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June 24: 1993 



To the Citizens of Oklahoma: 

This report provides an overview of the Oklahoma School Testing Program (OSTP) and 
summarizes students' average writing assessment scores from 1987 through 1993. The Writing 
Assessment Component of the OSTP was implemented in compliance with Title 70 O.S. Supp. 
1985,/1210.510 et. seq. (Senate Bill 426, 1986). 

Oklahoma has completed its seventh consecutive year of administering a direct writing 
assessment to students in tenth grade, and our sixth year of administering such a test to 
seventh'grade students. During the first five years of this program, writing in a combination 
of the narrative and descriptive modes was measured through use of the MAT-G Writing Test 
(The Psychological Corporation-TPC). 

In 1992, the State Board of Education adopted the Stanford Writing Assessment, Second Edition 
(TPC's newly normed writing test). This test introduced a more rigorous measure of writing 
skills, with scoring methods based on student expectations which include (1) directly 
addressing the topic of a unimodal prompt; (2) writing consistently within the specified mode; 
and (3) exhibiting a level of writing skill expertise which can reasonably be expected of 
students in the grade levels tested. The 1992 test elicited writing in the descriptive mode. 

Use of the Stanford in the 1993 OSTP Writing Assessment represented the state's first test of 
students' applied writing skills in the expository mode. Writing in this mode reqiiirRS students 
to provide an explanation of a concept, procedure, principle, or position (e.g., as on an issue). 

The majority of the 1993 seventh- and tenth-grade students scored closer to the national average 
than their predecessors who responded to narrative and descriptive writing assessment prompts. 
However, Oklahoma students continued to perform at levels above the national norm of the 50th 
Percentile Rank. Our teachers and administrators are to be commended for providing quality 
writing instructional programs. 

The State Department of Education appreciates the cooperation of all public school educators in 
the local management of the OSTP. In addition, we appreciate parents' assistance in preparing 
their children mentally, physically, and emotionally for participation in this program. It is 
only with such a spirit of school and community cooperation that we can successfiilly obtain a 
measure of our students' best efforts to reveal evidence of their accumulated knowledge and 
academic skills. 




"Sanfly Gan^t ^ 
Sti^ Supmntendent 
Secretary of Education 



2500 N. Lincoln Blvd., Oklahoma City, OK 73106-4699 (406) 621-3301, Fax: (405) 521-6206 
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OKLAHOMA SCHOOL TESTING PROGRAM (OSTP) 
WRITING ASSESSMENT COMPONENT 
1993 ANNUAL REPORT: EXECUTIVE SUMMARY 

The Stanford Writing Assessment was administered to Oklahoma's seventh- and tenth-grade 
students in February, 1993. Students' papers were hand-scored through use of two different 
procedures-holistic and analytic scoring. These procedures are designed to measure different 
aspects of composition skills. By using the combination of the two scoring procedures, we are 
supplying teachers with a variety of performance data that can be used to assess students' writing 
skill strengths, determine areas of skill reinforcement need, and provide focus for the most 
beneficial instructional methods for each student. 

Holistic scores provide an overview of the general quality of composition skills demonstrated by 
each student in response to a verbal writing prompt, Holistic Raw Scores range from 2 to 12, and 
are converted to normative scores to provide a comparison of student performance with that of the 
national norm group. 

The average Holistic Score Percentile Ranks for Grades 7 and 10 were 57 and 51, respectively. A 
Percentile Rank of 50 is considered the "national average." A score of 57, for example, means 
that the average seventh-grade student in Oklahoma scored as well as or better than 57% of the 
students in the national norm group who were administered this same prompt. 

Analytic Scores are given on a scale of one to four, with four being the highest performance level. 
The analytic scoring procedure for 1993 was designed to evaluate the six specific areas of writing 
skills listed below: (Figure 1, pages 20-21, provides further delineation of these skills. The table 
provided on page v describes Oklahoma's 1993 Analytic Score performance in terms of the skill 
area, performance level within each skill area, and the percent of students scoring in each 
performance level.) 

I. Ideas and Development IV. Sentences and Paragraphs 

II. Organization, Unity, and Coherence V. Grammar and Usage 

III. Word Choice VI. Mechanics 

Comparisons of Oklahoma students' Holistic Scores with the national average during the past 
seven years indicate continued above-average performance at both grades tested. During the first 
five years of this program, students were asked to respond to a combination picture/verbal prompt 
which elicited writing in narrative, descriptive, or a blend of these two modes. In 1992, a new test 
was introduced which is different in a number of ways, the most important of which include the 
following: 

• the prompts are strictly verbal (i.e., no pictures accompany the verbal portion); 

• the prompts are unimodal (i.e., elicit writing in only one writing mode); 

• scoring is more stringent in that the student is expected to (a) address the topic of the 
prompt throughout the discourse; (b) consistently write within the specified mode elicited 
by the prompt; and (c) use writing skills generally expected at the grade levels tested; and 

• national norms are newer, yielding more difficult standards with which to compare 
scores of ^dividual students and groups. 

Oklahoma students were administered unimodal prompts eliciting descriptive writing 
(1992) and, for the first time in OSTP history, expository writing (1993), Expository 
prompts differ greatly from narrative and descriptive ones in that such prompts require students to 
write an explanation of a concept, procedure, principle, or position (i.e., as on an issue). 

Prompts which elicit different writing modes are not statistically equated; therefore, scores on 
descriptive and expository writing assessments are not comparable. Use of the newly normed 
unimodal prompts has resulted in scores that are closer to national average performance. The 
shift to the new test with different writing and scoring procedures-along with an increased 
national level of writing achievement (especially at the high school grades, as indicated by the 
norms)-are possible reasons for differences seen in the 1993 and previous Oklahoma writing 
assessment performance comparisons with national norms. 
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OKLAHOMA SCHOOL TESTING PROGRAM (OSTP) 
WRITING ASSESSMENT COMPONENT: 



ANALYTIC SCORE SUMMARY-1993 



Grade 7 





I 

Ideas and 
Development 


II 

Organization, 
Unity, and 
Coherence 


in 

Word 
Choice 


IV 

Sentences and 
Paragraphs 
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Grammar and 
Usage 


VI 

Mechanics 


Score 


% of Students 
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% of Students 


4 (high) 
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Grade 10 





I 

Ideas and 
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II 
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Unity» and 
Coherence 


III 

Word 
Choice 


IV 

Sentences and 
Paragraphs 


V 

Grammar and 
Usage 


VI 

Mechanics 


Score 


% of Students 


% of Students 


% of Students 


% of Students 


of Students 


% of Students 


4 (high) 


17 


17 


15 


20 


13 


19 
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48 
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32 
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1 (low) 
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SUMMARY REPORT FOR THE 
OKLAHOMA SCHOOL TESTING PROGRAM (OSTP) 
WRITING ASSESSMENT COMPONENT-1993 



Introduction 

Beginning in 1986, Oklahoma Law mandated that a standardized, norm- 
referenced direct writing assessment be administered to students in specified 
grades. Tliis program— referred to as the Writing Assessment Component of the 
OSTP-began with administration of writing tests to students in Grade 10 in 1987. 
The program was expanded in 1988 to include assessment at Grade 7. 

The measurement instruments utilized to achieve an assessment of students' 
writing skills are obtained through the state bid system. Such instruments 
achieve this measure by providing a writing prompt to which students respond 
with an essay. Students' v^iting is hand-scored by three professional raters who 
apply two different procedures with each psLpev-holistic scoring and analytic 
scoring. Use of the former method results in a judgment of the paper's overall 
merit as compared with exemplary papers written during the national 
standardization of the tests, and yields the normative data required by law. 
Employment of the analytic scoring method yields performance estimates on six 
areas of specific writing skills. Both scoring methodologies are fully explained 
later in this report. 

The measurement instruments adopted by the State Board of Education for this 
program include the MAT-6 Writing Test (1987-1991) and the Stanford Writing 
Assessment Program (1992-93), both of which are published by The Psychological 
Corporation. While both test series assess students* actual writing skills, they 
differ in their philosophical bases and resulting methodologies. These 
differences will be discussed at length in a subsequent section of this report. 

The following report describes the inception, implementation procedures, 
assessment instruments and results of this program. District level data are 
reported for the 1993 assessment, and state level data are provided for the 
writing assessments conducted in 1987 through 1993. In addition, guidelines 
are provided for analysis and interpretation of the test results. 

A. Inception of the OSTP and the Writing Assessment Component: Legislation 

Oklahoma House Bill 1480, enacted in 1984, required the State Department of 
Education to develop a plan for statewide assessment of student achievement 
in the basic skills. This plan was submitted to the Legislature on January 1, 
1985. In 1985, the Oklahoma School Testing Program Act (§ 1210.505 through 
1210.511 of Title 70 of the Oklahoma Statutes) mandated the administration of 
a standardized, norm-referenced achievement test to students in Grades 3, 7, 



and 10 (House Bill 1466). The subject and skill areas measured in this 
assessment include reading, language arts, mathematics, science, and social 
studies. Data resulting from the OSTP are reported annually to the Oklahoma 
Legislature and the State Board of Education. 

The Oklahoma School Testing Program Act further required the State 
Department of Education to submit a plan for administering a direct writing 
assessment to Oklahoma's public school students. Submitted to the 
Oklahoma Legislature on January 1, 1986, this plan addressed (1) definitions 
of the various components of the wiiting assessment; (2) estimated costs; (3) 
timelines and procedures for implementation; (4) suggested tests to be 
utilized to provide consistent data; (5) reporting procedures; and (6) 
anticipated benefits to be achieved through the program. 

Oklahoma Senate Bill 426 (1986) created the Writing Assessment Component 
of the OSTP, mar\dating that a standardized, norm-referenced direct writing 
test be administered to tenth-grade students during the 1986-87 school year. 
This law further stated that, based on the availability of funds, the program 
would be expanded in 1987-88 to include Grade 7 and expanded again in 1988- 
89 to include Grade 3. (Subsequent legislation [i.e. Senate Bill 183, enacted in 
1989] deleted any reference to adding Grade 3; therefore, the program has been 
maintained at Grades 7 and 10 only.) 

Based on Senate Bill 426, the purpose of the writing assessment is to obtain a 
measure of students' actual v^iting performance; its function is to achieve 
improvement in the state public education system. Furthermore, "the State 
Board of Education shall seek to ensure that data yielded from the test is 
utilized at the school district level to prescribe skill reinforcement and/or 
remediation by requiring school districts to develop and implement a specific 
program of improvement based on the test results" (70 O.S. Supp., 1210.508). 
In addition, this bill stated that "the State Department of Education shall 
provide inservice training for local school personnel who administer the test." 

The legislation required that reports of test results be provided to local school 
districts by the contracting test company. Individualized results of the test 
"shall be made available by the local school district to the classroom teachers 
in the area tested" ...and "the Department will provide test results to 
appropriate public officials as required by law." 

Oklahoma House Bill 1110 (1987) expanded the Writing Assessment 
Component of the OSTP-with the same requirements-to include 
administration of a direct writing test to both seventh- and tenth-grade 
students. Subsequent legislation has maintained the current program. 
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Senate Bill 426 and House Bill 1110 further stated that students who have 
Individualized Education Plans (lEPs) pursuant to Public Law 101-476 shall 
not be required to participate in the Oklahoma School Testing Program. Due 
to effects of Federal Law (Section 504 of the Rehabilitation Act of 1973, 29 
U.S.C. 706, regarding the prevention of discrimination against the 
handicapped), the State Department of Education issued the following 
procedural directive to all public school superintendents annually through 
the school year 1988-89: 

"Based on Oklahoma Law, students who have Individualized Education 
Plans (lEPs) pursuant to Public Law 101-476 shall not be required to 
participate in the Oklahoma School Testing Program. However, the right 
of these students to participate in the OSTP cannot be denied. Parents must 
be given the opportunity to sign a form indicating their desire to have 
their children take the tests. The form to be used is the TEP Program SDE 
Form 9' which is employed during tb.e lEP Conference with parents. 
Please make sure that parents respond to the question on this form 
regarding their children's participation in the OSTP." 

Senate Bill 183 (Section 1210.511 B.), enacted in May of 1989, contained the 
following change in language regarding testing students with lEPs: 

"A student whose education is subject to the provisions of an 
individualized education plan (lEP) pursuant to Public Law 101-476 as 
amended, shall be tested pursuant to the provisions of the Oklahoma 
School Testing Program Act, Section 1210.505 et seq. of this title, only to 
the extent specified by the student's individualized education plan." 

With this change, the decision to test such students became a function of the 
lEP Team, of which the student's parent is a standing member. The "lEP 
Program SDE Form 9" continued to be utilized as the documentation of the 
team*s decision in regard to whether or not the student would be 
administer d the tests. 

Regulations were adopted by the State Board of Education in 1987 and 1989 to 
specify test exemptions for students who have lEPs and ^hose who are 
considered as Linuted English Proficient (LEP) based on the definition issued 
by the U.S. Secretary of Education. (A summary of the OSTP Regulations is 
found on pages 10 and 11 of this report.) According to Regulation, students 
who are considered as LEP can be exempted from OSTP participation at local 
district option (1) if such students meet the definition; (2) if these students 
have been counted as LEP for accreditation reporting purposes at the time that 
tests are administered; and (3) if parental permission for exemption from 
testing is obtained in writing and on file in the school. 



B. Combining Assessment Methodologies in the OSTP 

Two methods are available to measure writing and related skills-indirect and 
direct. Implementation of the OSTP Writing Assessment Component 
benefits teachers and students by providing data from both kinds of 
measurement. 

Indirect measurement is conducted through the administration of multiple- 
choice test items which consist of correct and incorrect optional answers 
illustrating the application of writing-related skills (e.g., word usage, sentence 
structure, punctuation, capitalization, grammar, and spelling). In such an 
assessment, the student must read the test item, decide which of the options 
is correct, and mark a machine-scorable answer blank to indicate his or her 
choice. The resulting scores do not indicate the student's ability to v^ite; 
instead, scores indicate the student's ability to apply knowledge of writing- 
related skills to printed sentences and phrases and choose the correct opdon. 
Students' strengths and deficiencies in skill knowledge/application are 
indicated by liie number of correct choices made in the measurement of each 
skill. Such data are reported on the language achievement tests administered 
as part of the OSTP. 

Direct writing assessment is conducted through the provision of a stimulus 
or prompt to which the student responds by writing a story, description, 
explanation, or other form of discourse designed to address the topic 
suggested by the prompt. Students* compositions are rated by professional 
graders. Rating of students' papers is based on predetermined criteria which 
represent specific writing skills and the level at which each is applied in 
the discourse. The direct assessment, therefore, provides an illustration of 
the student's actual ability to express his or her thoughts while demonstrating 
writing skills. 

The major purpose of the OSTP is to provide data with which to determine 
the status of studen.^' skill achievement levels. Its function is "to improve 
all phases of the state public educational system" (House Bill 1466). It is 
believed that tandem use of results from both components of the OSTP will 
ultimately function to increase the potential for improvement in public 
school writing programs. 

C. Use of the Standardized Writing Assessment Instrument: Benefits and 
Limitations 

A standardized writing test is one which has been constructed in accordance 
with detailed specifications as to kind and difficulty of prompt, time limits 
needed to complete the written response, and criteria utilized for scoring. In 
developing such tests, prompts are carefully constructed, field-tested with 
random samplings of students nationwide, and determined to be appropriate 
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in difficulty and discriminating power. The data resulting from 
standardization of such tests are used to establish a national "norm" or 
average against which students' scores at the individual, class, building, and 
state levels can be compared. Analysis of such test data reveals strengths and 
deficiencies in students' skills. Results of writing test score analyses can be 
employed in planning for instructional program improvement at all levels. 

1. Benefits 

The benefits derived from use of the standardized writing test include the 
following: 

• A special phenomenon transpires: in communities and schools when 
use of standardized tests expands from the local, district-controlled 
level to that of statewide assessment. In regard to writing skills-the 
last of the "basic skills" to be assessed with s*-andardized instruments- 
there is a renewed interest in (a) the skills being assessed; (b) the status 
of students' writing performance levels; (c) the need and desire for 
improvement; and (d) the methodology needed to achieve desired 
instructional goals. This interest— pursued with p>ositive action-results 
in renewed motivation of teachers and students, more appropriate 
instruction at all relevant grade levels, initiation of interim evaluation 
at the classroom level, and subsequent progress achieved with 
improvement of students' skills. 

• The standardized writing test is administered using the writing process 
method within a timed format, thus providing an estimate of the 
student's ability to organize and record thoughts in response to a 
specific stimulus. Considered as one of the "survival" skills, writing is 
applied in many situations throughout an individual's life. Such 
applications include written class assignments for which time is 
limited; teacher-constructed essay tests; job applications which require a 
timed impromptu autobiography, discussion of job-related goals, 

and /or description of leisure-time pursuits; and the on-the-job creation 
of correspondence, reports, and other business communications. Use 
of the standardized writing test and its results can reawaken the 
educator*s awareness of the student's need for success in such realistic 
applications. Reorientation to this factor often encourages educators to 
refocus instructional programs on students' specific skill needs, thus 
increasing the potential for long-range program improvement. 

• Use of the standardized writing test encourages educators to provide 
test-preparation sessions for students— a factor which helps individuals 
become more comfortable with test-taking. As teachers (a) help 
students learn to budget time; (b) counsel individuals privately on test 
results and their meaning for current needs and future success; and (c) 



reassure students through sharing progress reports periodically, each is 
motivated to take ownership in the testing situation and its outcomes. 
As students assume greater responsibility for their own learning, their 
efforts to improve the quality of their performance begin to increase~a 
factor which contributes to a more comfortable testing situation. 

Limitations 

Due to the structure of the standardized test and the specifications within 
\vluch each is constructed, administered, and scored, there are certain 
limitations which need to be considered in the analysis and use of 
resulting data: 

• Standardized test results represent estimates of the student's 
demonstrable achievement level only at the time the test was given 
and for a few weeks thereafter. With the passage of time, accumulation 
of additional knowledge and practice in writing, students' test scores 
can improve to a measurable degree. Therefore, neither individual 
students, school buildings, nor school districts should be labeled nor 
judged on a single test administrati:*n. 

• Failure to control certain necessary standardized test conditions can 
yield invalid results. Conditions which need to be controlled during 
test administration include (a) physical environment (room 
temperature, lighting, noise level, and overcrowding of students); (b) 
emotional environment (diminishing test anxiety and motivating 
students through positive attitudes on the part of parents, teachers, and 
school administrators); and (c) test administration procedures (giving 
directions accurately, observing strict timing of tests, and monitoring 
students' test-taking behaviors). Appropriate precautions regarding the 
control of all conditions mentioned above are discussed in the Pretest 
Inservice Training Sessions (and manual) provided for test 
administrators prior to testing. 

• It is neither equitable, meaningful, nor appropriate to compare the 
performance of individual schools nor school districts with one 
another without considering certain demographic factors which are 
known to exert influence on test results (Jenks, 1972; White, 1976). 
These factors include socioeconomic level of the community, median 
income, and median educational levels of community members. In 
addition, where small numbers of children are tested, the abilities of 
one or two students can affect the average score of any grade level of 
students in that district. 
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Furthermore, the transient nature of our society can have an effect on 
the overall performance level of the school district This factor-often 
referred to as student mobility rate, or population turnover rate— tends 
to exert a greater influence on scores of schools which enroll smaller 
numbers of students. 

In general, the higher the socioeconomic level of the community, the 
greater the median income (factors which correlate with educational 
level); and the higher the median educational level of the community 
is, the more likely the students in that school district are to score higher 
on tests. 

Within the time limitation imposed on the reporting of the enclosed 
results, the test data contained in this document could not be treated 
statistically to equalize the effects of those factors listed above. Under 
these conditions, it is both meaningless and inappropriate to compare 
the performance of students in specific schools or school districts to 
determine the effects of instructional programs. 

• Knowledge of test-taking skills can enhance the student's ability to 
perform on standardized tests. Such skills affecting the direct writing 
assessment include practice in responding to a writing prompt under 
tinied conditions, learning to organize the response before beginning to 
write, and editing/revising a piece of discourse either during the 
writing or immediately afterward. 

Current instructional theory advocates the use of the "process-writing" 
approach which includes the following steps: (1) pre- writing 
(organization and outlining of ideas); (2) writing the draft; (3) revising 
and editing; and (4) publishing. A search of the literature on writing 
instruction reveals that, for the past 15 to 20 years, this method has 
been proven to be a highly successftil approach. 

The Stanford Writing Assessment Program, (Second Edition), adopted 
for Oklahom.a's 1992 assessment, is designed to encourage students to 
utilize "process-writing" skills during the examination. Since this is a 
series of standardized, norm-referenced writing assessments, the tests 
are timed. Following directions in the manual, the test administrator 
guides students through a schedule which includes five minutes for 
the pre-writing stage; 20 nainutes for the writing stage; and the last 10 
minutes for the revising and editing stage. 

While the test adnunistrator provides oral directions to guide students 
through the timing of these three writing stages, students may elect to 
use the 35 minutes allowed for the test to construct the essay according 
to their own schedule. Performing well on such an assessment is often 
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largely deper\. .^nt on the degree to which the student has learned the 
"process-writing" skills and can apply them in this context. 

Teachers are encouraged to base daily instruction on the "process- 
writing" method, intensifying such instruction to the point of 
successful writing skill application by students. In addition, teachers 
are encouraged to provide frequent practice for students in prompt- 
initiated, timed writing sessions. The use of such methods-coupled 
with encouragement of an "ownership" attitude in students regarding 
writing practice and writing assessment-should serve to increase 
students' writing proficiencies. 

Implementation and Maintenance Procedures (1986-1993) 
A. Program Objectives 

Objectives for accomplishing the successful implementation of the 
Writing Assessment Component included the following: 

• To obtain a test which meets the specifications outlined by Oklahoma 
law, 

• To provide a system whereby tests remain secure. 

• To provide pretest inservice training for test administrators as 
outlined in the law. 

• To provide a system for dissemination, return, and accounting of test 
materials. 

• To provide a system for rating students' compositions-one which 
yields both normative data as well as data with which shidents' specific 
skill strengths and defidendes can be determined-and for reporting of 
results to local schools. 

• To analyze, interpret, and report statewide results to the State Board of 
Education and the Legislature. 

• To provide posttest inservice training on the interpretation and use of 
test results to improve instructional programs at the local school level. 

• To assist school personnel in incorporating the use of test scores in 
their School Improvement Plans. 



1 ^> 



B. Procedures Utilized to Accomplish the OSTP Objectives 

1. Obtaining a Standardized, Norm-Referenced Direct Writing Test through 
the State Bid System 

• In compliance with Oklahoma Law, a Request for Proposals (RFP) was 
developed and released to test publishers nationwide. This document 
provides a full description of the review and selection process to be 
employed; the major products and services for w" '.eh bids are being 
requested; timelines for delivery of materials and services; scoring 
systems which would provide the kind of data needed to erthance 
instructional improvement; and all other attachments required by law. 

• A "mandatory attendance" Pre-Bid Conference is held within two 
weeks after release of the RFP. All bidders' questions received prior to 
this conference and the Department's responses are submitted in 
writing to participants in this conference. All requirements of the 
Writing Assessment Component of the OSTP are discussed and bidders 
are allowed to clarify any misconceptions regarding bid requirements. 

• The test review process is designed to obtain an objective evaluation of 
all companies' bids for products and services in terms of the manner in 
which the following requirements— either the Department's or those of 
the law— will be met: (1) test specifications; (2) scoring criteria and 
methodologies utilized; (3) security of test materials and individual 
students' score reports; (3) SDE staff training for pretest and posttest 
inservice programs; (4) provision of inservice materials (i.e., in terms 
of quality, quantity, and deadlines); and (5) customization of formats 
designed for testing and reporting in a manner that best meets the need 
for instructional program improvement. 

Evaluation forms consisting of more than 100 criteria are utilized by 
review /evaluation committees to judge the appropriateness of products and 
services bid by companies. One form is completed by each reviewer to 
indicate his or her evaluation of each company's set of proposals and test 
instruments submitted. 

For evaluation of writing test bids, more than thirty (30) junior high and high 
school composition teachers, coimselors, school administrators, testing 
experts from public schools, and university professors having expertise in 
either tesl-s and measurements and/or writing instruction serve as the 
evaluation committee. During the evaluation process, committee members 
spend at least 2.5 hours studying the proposals and tests bid, after which 
company representatives conduct two-hour presentations that include 
answering all questions raised by reviewers. 




One of the major factors the review committee is asked to consider is the 
degree to which the tests submitted by bidders measure Oklahoma's 
Suggested Learner Outcomes (SLOs) for composition. Considered as equally 
important is the degree to which scoring criteria and techniques (1) reflect 
measurement of the SLOs, and (2) provide information with which teachers 
can determine and remediate students' specific writing skill deficiencies. 
(Section m-C of this report [Figures 2 & 3, page 25-26] describes the match of 
measurement techniques with Oklahoma's SLOs and the manner in which 
such techniques produce results which relate to each of the measurable 
Learner Outcomes. Section m-C of this report describes the specific scoring 
criteria used to rate Oklahoma's students' compositions [holistic scoring, 
pages 15-18; analytic scoring, page 19.) 

Responses on the evaluation forms are tallied, revealing the committee's 
recommendation. The Stcte Board of Education considers all proposals, 
studies the evaluation committee's recommendation, and adopts the 
appropriate program. 

2. Providing for Maintenance of Test Security 

Maintenance of test security has become a major issue-both nationally 
and in Oklahoma. The test security measures previously outlined in State 
Board Regulations were greatly enhanced during the 1989-90 school year. 
Additional security measures implemented in 1989-90 and maintained 
throughout 1993 are outlined as follows: 

• The test form being utilized is embargoed by the test publisher so that 
no person living in Oklahoma can obtain a copy other than through 
standard OSTP distribution at the time of testing. 

• Test booklets are delivered to schools in sealed envelopes which are to 
be opened at the beginning of the test session only by students who are 
to be administered the test. All unused tests are to remain in sealed 
envelopes. 

• Test materials are to remain locked in a secure area until test 
administration begins, during all hours between test sessions, and until 
such documents are returned to the test publisher for inventory, 
scarming, and scoring. 

• An adult monitor other than the test administrator is required to 
witness the test administration and securing of test materials. 
Selection of monitors is subject to the approval of the district 
superintendent, elementary superintendent, or building principal. 
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• Security forms are provided for school superintendents, principals, 
building and district test coordinators, test administrators, and 
monitors. These forms state that, to the best of the participants' 
knowledge, the specified test security measures were followed. All 
forms are to be signed by the appropriate OSTP participants and 
retiimed with answer documents. An accounting of these signed 
forms is documented by the State Department of Education. 

• Adult participants in the OSTP are directed to refrain from overtly or 
covertly exposing students to the test items and/or answers to test 
items. 

• Violations of test security are to be reported to the State Department of 
Education, investigated, and proven incidents can result in such action 
by the Board as invalidation of the sdnocl's test scores and/or 
revocation of the perpetrator's teaching, counseling, or administrative 
certificate(s). 

Providing Awareness and Instruction for Local School Personnel (1986-93) 

• Announcements of legislation regarding the implementation and 
maintenance of the Writing Assessment Component of the OSTP are 
accomplished through notices placed in the Administrative Focus and 
the Results In-Class in late Spring or early Fall semesters. Such 
notices delineate the legislation and its intent; describe the kind of test 
which will be administered; announce test dates and the options 
offered to school superintendents for choosing available dates; specify 
the population to be tested and provide guidelines for testing Limited 
English Proficient (LEP) students and those with Individualized 
Education Plans (lEP); and announce the Department's plan for 
offering pretest inservice training for all schools. 

• A letter is distributed to all school superintendents early in the Fall 
semester each year. Test dates and procedures are again delineated. 
Accompanying this letter is a survey to obtain data on the number of 
students to be tested, the number of test administrators, the names of 
building and district test coordinators, and the proposed test dates for 
each school district. 

• Pretest inservice training sessions are conducted for all school districts 
during January and early February each year. Topics for instruction 
include (1) description of the program; (2) implementation procedures 
and policies-with emphasis on maintaining test security; (3) 
specification of test environmental conditions necessary to 
maintaining test validity; (4) test administration procedures; (5) 
scoring procedures and criteria; (6) descriptions of report formats; and 
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(7) procedures for handling test materials (inventory, distribution, 
coding, packaging, mailing, etc.). All inservice participants receive a 
procedural manual which includes specific instructions for building 
and district test coordinators and test administrators. 

4. Providing Materials and Services 

Test materials are distributed to school superintendents by The 
Psychological Corporation. Such materials include test booklets, teachers' 
manuals for test administration, and the district test coordinators' kits 
containing all materials and instruction for packaging and returning of 
materials to The Psychological Corporation. 

5. Administering the Tests 

Tests are administered on a Tuesday, Wednesday, or Thursday, based on 
each local school superintendent's choice of date selected from three 
specified weeks in February. Students' test booklets and other materials 
are returned to The Psychological Corporation immediately following 
administration of the tests. 

6. Scoring and Reporting 

• Students' papers are hand-scored by a minimum of three professional 
raters per paper at The Psychological Corporation's Writing Scoring 
Center in San Antonio, Texas. Part n (pages 15-23; 61) of the Manual 
for Interpreting describes the procedures used to train raters and score 
Oklahoma's papers. A copy of this manual may be obtained from the 
Student Testing Section, State Department of Education (405) 521-3341 

• Results of the tests are disti-ibuted to school superintendents in April 
each year. Part m (pages 25-44, Manual for Interpreting) provides 
information on interpreting test results. 

m. Description of Materials; Testing, Scoring, and Reporting Methods; and 
Related Services 

A. Materials and Related Services 

The Stanford Wr itin g Assessment Program is a nationally normed direct 
writing test. The format focuses on an exclusively verbal prompt which 
elicits writing in one of four writing modes: narrative, descriptive, 
expository or persuasive. The expository writing mode was selected for 
use in Oklahoma for the 1993 assessment. 
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Test materials include (1) a separately sealed page containing the writing 
prompt; (2) a student response booklet which contains one nonscorable page 
for prewriting activities such as planning and /or outlining, and four lined 
pages for writing; and (3) a test administrator's manual containing directions 
for the examiner to read orally to students. Based on HB 1946, enacted in 
1990, these materials are not distributed to test administrators until the 
morning of the day testing begins. As the test session begins, student test 
materials are distributed, and each student unseals the writing prompt page, 
listens to directions, and begins writing. 

A District Test Coordinators' Kit is distributed approximately two weeks 
before the test materials are received. This kit contains all materials and 
instructions necessary for efficient management of the testing program in the 
local school. 

Pretest inservices are held during the month before testing for every school 
district in the state. State Board Regulations require that at least one 
representative from each school district attend these inservices, and manuals 
are distributed with oral instructions for conducting the assessment according 
to the standardization practices utilized to norm the tests. 

Posttest inservices are conducted throughout the year for all individuals or 
groups who request these services. These sessions focus on interpretation 
and use of test scores to improve ins*- ictional programs at the individual, 
class, building, and district levels. A opy of the Manual for Interpreting is 
provided for each person who attends such inservices; two copies are 
distributed to each school district superintendent with the score reports. 

B. Testing Methods 

Standardization procedures require that students begin the testing session 
with at least two sharpened #2 pencils with erasers. Students are not allowed 
to have extra paper on their desks when the test begins. However, if students 
need additional paper for completing an essay, the teacher is allowed to 
supply the amount of paper needed upon the student's request. 

Materials are distributed, and students complete the school, district and 
student identification information on the front of the response booklet. 
Students then open the sealed prompt. The test administrator—following 
directions ver batim in the official Test Administration Manual—reads the 
prompt orally to the class (two times) while students read the prompt silently. 
Students are then guided through a schedule which permits five minutes for 
prevailing (i.e., planning, outlining); 20 minutes for v^iting; and 10 minutes 
for revising and editing. 



This mc thod of testing is different from that utilized in the MAT-6 Writing 
Test which Oklahoma adnainistered from 1987 through 1991. In using the 
MAT- 6. students were given directions and then allowed 20 minutes to 
complete an impromptu-writing exercise. With the Stanford, students are 
tested in a manner that applies the writing process, the instructional method 
most strongly advocated in research during the past 15-20 years. The writing 
process as an instructional method is advocated in Oklahoma's Suggested 
Learner Outcomes published in 1983 and in the newly mandated Learner 
Outcomes published in 1991-92, This method features pre- writing 
organizational strategies, writing, revising, editing procedures and 
publishing. 

The timed impromptu writing and timed process-xvriting methods of 
assessment and instruction each serve a distinct purpose which fulfills 
writing skills needed both in school and in the work place. However, the 
assessment procedure used in the Stanford supports a writing method which 
assists students in learning to organize information before writing, and to 
edit /revise their draft copies to eliminate errors. The learning and 
application of such skills results in improved discourse created for any 
writing purpose. 

C. Scoring Methods and Reporting Services 

Students' essays are scored by professional writing assessment raters at The 
Psychological Coiporation. Scoring of each paper is conducted through use of 
two methods— fto/istzc and analytic. These methods are designed to evaluate 
different (but complimentary) asp>ects of students' writing skills. 

Each response booklet contains an identification number which is associated 
with the student's name. Before scoring is conducted, the page containing the 
student's name is removed from the remainder of the response booklet 
Therefore, raters who score the papers are not aware of students' names, 
gender, or any other identifying information. 

To assure a high level of reliability and validity in scoring students' papers, 
rater reliability statistics are established throughout the scoring process. The 
scores assigned papers by each rater are recorded on a computer as each rater 
completes a set of 25 papers. Each hour, the rater is checked for accuracy by 
having another trained rater validate the scores being assigned by that person. 
If a rater's quality level begins to fall short of an established criterion, the rater 
ceases his or her work for the day. Use of this form of quality control results 
in strong reliability estimates-a factor which is necessary to the production of 
comparable scores. 

Both holistic and analytic scores are reported to school districts at the 
individual student, school site, and district level, A separate report is 
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provided for the parents or guardians of each student. In addition, the State 
Department of Education is provided with group level aggregated scores for 
school sites, districts, and the state. The State Department of Education 
refrains from collecting or filing any OSTP test data which has a student's or 
teacher's name attached* 

The following scoring methods are used to rate each student's essay: 

1. A holistic scoring rubric which focuses on the overall merit of the paper is 
utilized to rate each essay. The major factors considered in holistic scoring 
include (a) how well the paper is organized (i.e., does it have a good 
beginning, development, and conclusion); (2) how well the concepts 
introduced are clarified and supported; (3) attention to audience in terms 
of appropriateness of language and content; and (4) fluency— the extent to 
which the writer uses syntax, word choice, and mechanics to help the ideas 
flow smoothly from beginning to end. 

During the standardization (or national norming) of the tests, sets of 
anchor papers were selected. Anchor papers demonstrate characteristics 
typical of a particular score point for a specific prompt. These anchor 
papers were written by students in the grades for which each prompt was 
being normed and represent examples of the quality of writing expected 
for a paper to be assigned each of the scores in the holistic scale. Anchor 
papers are then utilized for comparisons during the rating of papers to 
assure consistency in assigning the various scores. 

The Stanford Writing Assessment was normed with the use of writing 
skill standards (or expectations) established at four levels of schooling. As 
one would exp>ect, the standards are dependent on the writing mode 
elicited by the prompt. 

The expository mode was used in Oklahoma in 1993 at both 7th and 10th 
Grades. The standards used in the holistic scoring of papers written in this 
mode are described as follows: (excerpted from The Manual for 
Interpreting, The Psychological Corporation, 1993) 

Standard I, Grades 3 and 4 

Especially at the lower grade levels, students generally are not expected to 
produce an impressive amount of e):pository text. This realistic 
expectation is based on two important considerations. First of all, third 
and fourth graders have not had a great deal of experience in exposition. 
Second, the abstract nature of the task requires more reflection than some 
other types of wrriting do. Therefore, it is not surprising that students 
present only a moderate number of reasons and supjx)rting details. 



The most obvious feature of exposition at these levels is its list-like 
quality. This charact ^'istic is not limited to the least successful papers. 
Many of the more corxipetent responses consist of a glorified list of 
reasons. In adddition, as students try to structure their ideas, they 
sometimes add extraneous material from another mode (usually 
narrative). 

In terms of organization, third and fourth graders tend to generate loosely 
joined content. For example, a paper in the middle range in this mode 
may not have a well-defined organizational strategy. It may simply 
demonstrate some logical structure in presenting a number of reasons. 
Transitions are usually weaker and less varied than in the expository 
prose of older and more mature students. 

At the upper score points in particular, writers generally use some specific 
rather than only general words. However, specificity in this mode (and in 
persuasion) does not always possess the "physical concreteness" found in 
other modes. A student presentation that seems rather vague may, in fact, 
be an informationally rich sequence of ideas linked by cause and effect. 

Students also occasionally use subordinate clauses that are appropriate to 
the task. Third and fourth graders may be relatively unfamiliar with 
exposition as a type of writing, but most students naturally employ 
''because*' clauses in their everyday speech. This practical knowledge of 
syntactical complexities can help a great deal in a mode whose challenges 
students have just begun to explore. 

Standard II, Grades 5 and 6 

In terms of both intellect and experience, most fifth and sixth grade writers 
are able to generate a respectable quantity of expository text. Therefore, 
while the tendency to list by no means disappears, a higher percentage of 
students present details that flesh out assertions. In addition, the amount 
of off-task information decreases, especially at the upper score points. In 
papers that earn the highest scores, students stay on course as they attempt 
to amplify their ideas. 

A number of students in the upper elementary grades have been exposed 
to the rudiments of expository structure. They have learned how to state 
their thesis in an opening paragraph, explain their reasons in the body of 
the paper, and conclude with several lines that re-emphasize the 
compositions' s controlling idea. As a result of such instruction and of a 
growing ability to think abstractly, fifth and sixth graders are usually more 
in control of the pr?sentation than younger students are. Some degree of 
disorganization is seen in some papers at the lower score points, but all 
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better-than-average essays exhibit an organizational strategy. The best 
papers use temporal transitions C'when/' ''after*'), logical connectives 
("first," "finally") or an appropriate combination of both. Students in fifth 
and sixth grades also respond with a greater degree of specificity. 
However, a significant percentage of writers at these grade levels continue 
to have difficulty with a sustained specific discussion. For this reason, 
quite a bit of the information is general. Superior pieces, however, 
contain a few concrete examples and /or detailed reasons that are precise 
illustrations of cause and efect. A corresponding increase of syntactic 
subtlety, particularly in the use of complex sentences, is also seen. Such 
complicated structures are usually necessary to express sophistication of 
thought. 

Standard III, Grades 7 and 8 

It is at the middle school level that students develop a flexibility in 
writing. Still very much at home in the world of immediate experience, 
they are fairly secure in the realm of abstraction as well. They understand 
in a much more comprehensive way than younger students the 
relationship between complex ideas. In addition, they are capable of a 
passionate commitment to a position that is rare at the lower grade levels. 
These cognitive developments, plus continued practice in the expository 
mode, allow for developmental leaps in expectations at Grades 7 and 8, 

More so than at earlier grades, writers begin to be able to sort out wheat 
from chaff. That is, they select content that is relevant to the task. The 
"inventory" approach so characteristic of younger composers begins to 
recede. Essays receiving top scores contain extended discussions of major 
points. Successful writers flesh out examples in greater detail and/or 
carry if- then reasoning a step or two further. 

Evidence of organizational strategy extends even into the lower score 
points. Extremely disorganized papers are not common. Highly 
successful student writers are sure-handed from start to finish. In order to 
cieate a tightly woven piece, authors sometime employ a variety of 
temporal transitions and logical connectives. 

Specificity is increasingly important as students attempt to expand the 
presentation. Of course, some generality is expected. However, lengthy 
and fluent responses without specif icty are not usually seen at the higher 
score point ranges. In terms of syntax, simple sentences are used sparingly 
and effectively. As writers struggle with the intricacies of why and why 
not, of implication and example, the skillful coordination and 
subordination of ideas become crucial. 
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standard IV, Grades 9 Through 12 

High school students are well prepared to ineet tl\e rigors of exposition. 
Their conceptual powers have increased tremendously since middle 
school, and most have had opportunities to write in the expository mode. 
Many of the earlier obstacles to effective expression have been removed. 
Therefore, students at these grade levels do not generally produce chunks 
of extraneous material or retreat from the task by giving a list of reasons 
with only a few supporting details. Writers typically generate a significant 
amount of expository text. 

For example, even a paper in the middle score-point range contains some 
development. That is, it offers a sustained discussion of at least one key 
point Presenting layers of illustration and relentless logic, the most 
competent authors state their case in a definitive, adult-like fashion. 

Writers at these grade levels organize their papers tightly. Most high 
school students have been taught the basic structure of the five-paragraph 
essay. They know both how to establish a dear direction and how to 
employ different kinds of transitions that help the reader move from idea 
to idea. At the upper score points, only occasional lapses are evident. 
Some of the best writers employ original organizational strategies. 

An increase in verbal and syntactic sophistication is common as well. 
Some precision is evident even at the lower score points, and the best 
papers exhibit a high degree of specificity. Candidates for the highest 
scores are adept at using varied sentence structure that is appropriate to 
the subtleties of cogent reasoning. 

Holistic scoring is conducted by two raters who independently read 
students' essays, each assigning a score of one through six. Neither rater 
knows the score assigned to a given paper by another rater. The scores are 
recorded on a computer and those scores on a given papor which are more 
than one score point apart are identified. Each such identified paper is 
rated by a third person who, by design, is not aware of the scores assigned 
by the first two raters. Scores are again compared, and, at this point, if 
there are not two resulting adjacent scores, the paper is rated by another 
resolution reader. The final assigned raw score of 2-12 represents the sum 
of the two adjacent scores assigned the paper. 

The holistic score scale is utilized to establish normative data (i.e., 
percentile ranks, stanines, and normal curve equivalent scores). It is the 
score with which an individual's or group's performance can be compared 
with a national and/or state norm. In addition, local district noimative 
scores are provided for school districts which enroll more than 2200 
students in grades K-12. 
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2. An analytic scoring rubric is also used to rate each essay. Analytic scores 
focus on six specific categories of writing skill: Ideas and Development; 
Organization, Unity, and Coherence; Word Choice; Sentences and 
Paragraphs; Grammar and Usage; and Mechanics. Raters assign a score of 
one through four within each category to indicate the relative strength of 
the student's specific v/riting skills. Figure 1 on pages 20-21 of this report 
displays the specific skill levels associated with the scores assigned within 
each of the analytic score categories. 
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HGUREI 

ANALVnC SCORING GUIDELINES 

AREA I: Ideas and Development 

4 Extensive development of several ideas (or one, main important idea) with 
extension and elaboration on all or most of the points. Look for uniqueness, 
interest to audience, and strong supporting details. Can be exceptional 
writing or extremely thorough. 

3 Good development of idea with many details extended and elaborated. Ideas 
are fairly well supported. May be an extensive list vdth little or no 
elaboration. 

2 Adequately supported ideas with some details extended or elaborated. 

1 Weak ideas minimally supported vAth little or no extension of details. 
AREA II: Organization, Unity, and Coherence 

4 Completely organized with smooth flow from one idea to the next through 
the use of transitions and sequencing. Unity is strongly evident with no 
wandering from primary theme or plan. 

3 Fairly well organized vdth good unity of plan. Some transitions may be used. 
Little or no digression from main ideas or writing mode. 

2 Small amount of organization. Weak plan that may not be well unified. 
Ideas may be only minimally connected. May frequently wander from 
expected writing mode. 

1 Lack of plan or coherence. 
AREA III: Word Choice 

4 Precise, appropriate, accurate, and specific word choices that convey the 
correct meaning and appeal to the audience. May be vivid and imaginative. 

3 Word choices that are appropriate and specific. May lack "sparkle,'' but 
meaning is clear. 

2 Fair use of words. May be specific but have little variety. May be simplistic 
and occasionally vague, but generally effective. 

1 Very simplistic. Meaning may be unclear or inappropriate. 
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ANALYTIC SCORING GUIDELINES (continued) 
AREA IV: Sentences and Paragraphs 

4 Excellent control and formation of sentences. Variety of sentence structure, 
type, and length contribute to fluency and interest. Paragraphs used where 
appropriate. Few, if any, errors. 

3 Adequate control of sentence formation. Some mix of sentence types, 
lengths, and structures. May contain a small number of errors that do not 
interfere with fluency. May have some attempt at paragraphing. 

2 Most sentences constructed correctly. May have little variety in type, length, 
or structure. May be somewhat monotonous or choppy. May contain several 
errors or lack control. May have no attempt at paragraphing. 

1 Poor sentence structure with many errors that inhibit fluency or clairity. 
Lacks control 

AREA V: Grammar and Usage 

4 Error free or very few errors in approximate proportion to length of paper. 

3 Good grammar and word usage. Errors that do not detract from overall 
quality of the paper. 

2 Fair grammar and usage. Errors may interfere with meaning. May be 
simplistic. 

1 Poor grammar and word usage with frequent or serious errors. 
AREA VI: Mechanics 

4 Error free or very few errors m approximate proportion to the length of the 
paper. 

3 Most punctuation, capitalization, and spelling correct. Errors do not interfere 
with communication. 

2 Some errors in punctuation, capitalization and/or spelling that may interfere 
somewhat with communication. 

1 Frequent and/ or serious errors in mechanics that interfere with 

communication assigned within each of the analytic score categories. 
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Raters who conduct the analytic scoring look for certain characteristics of 
good writing within each of the categories of skills listed above. The rater 
looks not only for correctness, but also for complexity, sophistication, and 
a variety of writing techniques. The following descriptions are excerpted 
from the Manual for Interpreting (The Psychological Corporation, 1993) 
which will be used as Oklahoma*s posttest inservice manual: 



• Area I: Ideas and Development 



In this category, readers pay close attention to the amount of quality 
information that the student presents. Has the writer added layers of 
detail that enrich the discussion of key points to the essay? Is tiiere a 
minimum of bare assertion (statements that receive very little, if any, 
clarificatior.)? Are the ideas interesting, original, and tailored to the 
audience, whether generalized or specific? 

• Area II: Organization, Unity, and Coherence 

Raters scoring papers in this category look for evidence of an overall 
organizational pattern. Is the student in control of the presentation 
from start to finish? Are transitions used effectively to connect ideas 
both within and between paragraphs? Does the writer stay on course by 
avoiding redundancy and out-of-order material? 



• Area III: Word Choice 



Within the category of Word Choice, raters look for the kind of specific 
and vivid vocabulary that makes writing come alive. Is there a 
sustained attempt on the students' part to use active verbs and precise 
modifiers that go beyond bland generalities like "get," "nice," and 
"fun"? Does he or she offer on occasion a figure of speech (such as a 
metaphor or siinile) that adds a poetic touch to the description? 



• Area IV: Sentences and Paragraphs 



As readers score responses in this category, they look for complex 
sentence structures that enhance meaning. Are sentences structurally 
complete, smoothly written, and clear? Do sentences vary in length 
and complexity? Is a variety of sentence patterns used? 



• Area V: Grammar and Usage 



In this category, readers look for the correct application of grammatical 
principles. Does the writer use correct subject-verb agreement and 
logical pronoun i*eference? Does he or she continue to use the tense 
established in the beginning of the essay? L every variance from the 



22 3c' 



tense appropriate? Does the student show an adequate knowledge of 
typical problems (such as the use of "them" instead of "those")? 

• Area VI: Mechanics 

When evaluating student writing in this category, readers look for 
consistently correct capitalization, punctuation, and spelling. Is a 
variety of capitalization and punctuation skills used correctly? Is the 
student occasionally successful in using such risk-taking techniques as 
ellipses and dashes to show complexity of thought? Does correct 
spelling enable raters to move smoothly from sentence to sentence? 

Analytic score results are provided for parents and teachers on the 
score report. Such information is useful in providing appropriate 
instruction for specific writing skill improvement, and assists teachers 
in fulfilling the intent of Oklahoma law. 

Holistic and analytic scoring are conducted for different purposes—the 
former, to provide an overall rating of the paper, and the latter, to 
provide a view of the strengths and weaknesses of a student's specific 
writing skills. While the two methods produce different views of 
writing skill, there is a relationship shared by the two methods. This 
relationship is discussed in Section IV of this report. 

D. The Match with Oklahoma's Suggested Learner Outcomes (1983 ed.) 

As stated previously, one of the major criteria used in selecting an 
instrument for the Writing Assessment Component of the OSTP is the 
manner in which the test measure Oklahoma's Suggested Learner Outcomes 
(SLOs) for composition. Figures 2 and 3 (pages 25-26) illustrate the match of 
SLOs in grades 7 and 10, resf)ectively. Information contained in Figures 2 and 
3 is explained as follows: 

• Each Suggested Learner Outcome is given in its full text as worded in the SLO 
Guide. Codes in the extieme left column match the coding of each outcome 
within the SLO Guide. 

• Listed in the column labeled "HOLISTIC" are indications of whether or not 
each SLO is measured by the holistic grading procedure. "Yes" indicates that 
the SLO was measured; "NA" indicates that the SLO could not be measured 
by this method. 

• Listed in the column labeled ' ANALYTIC" are indications of whether or not 
each SLO is measured by the analytic scoring procedure. "Yes" indicates that 
the SLO was measured by this method; "NA" indicates that measurement of a 
given SLO was not applicable. 
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Provided in the extreme right coliimn of Figures 2 and 3 is an explanation 
regarding the manner in which measurement of a given SLO is addressed by 
each of the rating methods. 
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IV. Understanding and Using the Data 

A. The Comparison of Holistic Scores with National Norms 

Oklahoma's group scores are reported in terms of the AVERAGE score for a 
specific group of students (i.e., building, district, or state). The holistic scores 
are converted to national norms (e.g., National Percentile Ranks, Stanines, 
and Normal Curve Equivalents [NCE] ) to provide a comparison of our 
students' performance levels with those of the national norm group. 

The national norm group is comprised of students whose selection is based 
on a scientific random sampling. The goal of such sampling is to achieve 
grade level groups, with each group representing a cross-section of the United 
States regarding an appropriate balance of geographic areas, socioeconomic 
levels, ethnic groups, urban/rural locations, and other critical factors.. 

The norming (or standardization) process involves the administration of 
appropriate test forms to students comprising each grade level national norm 
group. Each group's average raw score on a given test at a particular grade 
level becomes the "norm" with which performance of individuals or groups 
of students in the same grades can be compared. (Example: the average raw 
score of students in the national norm group is represented by the 50th 
Percentile Rank, the 50th NCE, or a Stanine of 5 on a particular test. If 
Oklahoma's 7th grade students' average score is at the 57rd Percentile Rank 
on a particular test, it can be stated that the "average" 7th grade student in 
Oklahoma is performing at a level that is equal to or better than 57% of the 
students who comprised the national norm group for that test.) 

The Stanford Writing Assessment Program was normed concurrently with the 
Stanford Achievement Test Series, Eighth Edition. Norming took place in the 
spring and fall of 1988. Data were collected on both forms, J and K, at each of the 
nine levels of the Stanford Writing Assessment Program . Equating studies 
linking forms and levels of the assessment were also carried out during 
standardization. Within a mode only, prompts may be compared with one 
another across grades/levels (e.g.. Advanced 1, Grade 7 • TASK 2, Grade 10) and 
within a grade /level (e.g.. Form J - Form K). The demands on the writer of the 
different modes are great enough that cross-mode comparisons are not 
recommended. 

Oklahoma's seventh-grade students were administered the Advanced 1 level. 
Form K of the Stanford Writing Assessment; a level specifically desigi^ed for the 
spring of grade 7. The tenth-grade students were administered the TASK 2 level. 
Form K of the Stanford . Form K was used at both grade levels because this form is 
secure. 
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The Stanford Writing Assessment offers writing prompts at each level that elicit 
the four major types of writing: Narrative, Desaiptive, Expository, and 
Persuasive. In 1993, Oklahoma students at Grade 7 and at Grade 10 responded to a 
expository writing prompt. 

There are several features of the new Stanford Writing Assessmen t that 
distinguish it from the MAT'6 Writing Tost used previously. Figure 4 (page 29) 
compares the two writing tests. 
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Figare4 



ACOMPARISON OF MAJOR FEATURES OF THE 
MAT'6 Writing Test and the Stanford Writing Assessment 



rEATUKE 



MAT^ Writing T^>st Stanford Writing Agsefisment 



Establishment of 
National Norms 

When Used in Oklahoma 
Ty pe of 

Writing Prompt 

Writing Method 
Measured 



Time Allowed for Testing 
(not including distribution 
of papers and giving 
directions) 

Writing Modes 
Elicited in the Tests 



Mode(s) Used in 
Oklahoma 



Holistic Scoring 



Holistic Scoring ScaJe 



Scoring Ranges 
(re: Holistic only) 



1985 

Grade 7: 1988-1991 
Grade 10: 1987-1991 

Combination verbal 
and picture 

Timed impromptu 



Analytic Scoring 



20 minutes 



Each prompt elicits a 
combination of 
narrative and 
descriptive writing. 

Narrative/Descriptive 
in 1987-1991 

Allows for either or 
both modes to be used 



Scored 1 through 8, 
yielding raw scores 
of 2 through 16. 

Use of one scoring 
standard for both 7th 
and 10th grades. 



Four skill areas; four 
performance levels for 
each skill area. 



1988 

Grades 7 and 10: 1992 and 1993 
Exclusively verbal 



Timed, modified "process" 
writing, with teacher 
instructions guiding students 
through the various steps of the 
writing process. 



35 minutes 

Each prompt is mode-specific; 
available in narrative, descriptive, 
expository, and persuasive modes. 

Exclusively descriptive (1992) 
Exclusively expository (1993) 

Mode-specific (any writing not in 
the specified mode elicited is not 
considered for holistic scoring). 

Scored 1 through 6, yielding 
raw scores of 2 through 12. 



Use of different developmental 
writing stage standards-one 
for 7th grade and one for 10th 
grade. Provides for more accurate 
comparisons of writing within 
smaller age/grade ranges. 

Six skill areas; four performance 
levels for each skill area. 
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There are three features of the Stanford Writing Assessment that are new for 
Oklahoma and have a direct impact on the holistic scores and the national norm- 
referenced scores earned by students: 



1. The entirely verbal prompt on the Stanford is scored in a mode-specific way. 
In 1993, Oklahoma students responded to an expository prompt. Unlike the 
MAT-6 Writing Test that would score any mode of writing the student 
produced that was related to the prompt the Stanford requires an expository 
writing to an expository prompt in order for the writing to be scored. This 
scoring requirement for mode-specific writing means that a lengthy 

essay which has, for example, long unrelated narrative or descriptive 
sections, would receive a holistic score based only on the expository sections 
of the essay. Mode-spedfic scoring applies in each of the four writing modes 
tested by the Stanford . 

2. The Stanford Writing Assessment rates each essay on a holistic scale thit 
ranges from 1 (low) to 6 (high). Two readers independently rate each essay 
and their scores are added together. Thus, the final score for a paper can range 
from 2 to 12. This contrasts with the MAT-6 Writing Test scoring in which 
each essay was rated on a scale frDm one to eight, yielding a raw score from 
two readers that could range from 2 to 16. The shorter Stanford scale is 
accurate and is more like that used across the nation. It also means that a 
greater range of percentile rank points are tied to each raw score point on the 
Stanford, than on th e MAT-6 . 

3. On the MAT-6 Writing Test there was one set of rules for judging the holistic 
(overall merit) scores for seventh and tenth grade papers. On the Stanford, 
there are four sets of rules or scoring standards for each type of prompt; one 
each for primary (Grades 3-4), intermediate (Grades 5-6), junior high school 
(Grades 7-8), and high school (Grades 9-12). Oklahoma's seventh grade 
students' papers were judged using the standai-d appropriate for junior high 
school students. This means that on the Stanford, writing is judged against 
developmentally appropriate standards. 

National norms are available only for comparing students' holistic scores, as 
this scoring technique was the one used in establishing norms during the 
standardization study. The specific criteria used to apply the analytic scoring 
method to Oklahoma students' papers were customized for use in 
Oklahoma's program. This was accomplished to satisfy Oklahoma's request 
for data to use in (1) identifying students' writing skill needs; and (2) planning 
and implementing appropriate programs to accomplish the desired writing 
skill improvement. Therefore, subsequent discussion of "norms" in this 
report have reference only to students' holistic score averages. 
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As explained earlier in this report, two raters independently rated Oklahoma 
students* papers on a holistic scale of 1-6. If the scores given by two r Jers 
were more than one point apart from each other, a third rater (a seasoned 
team leader called a resolution rater) scored the paper. Each of the original 
scores was then compared with the third score, and the two closest scores 
were added to yield the holistic raw score (provided that the two closest scores 
were no more than one point apart). Thus the range of possible holistic raw 
scores was 2-12. 

Tables 1 and 2 on page 32 detail the raw scores, scaled scores and derived 
norm-referenced scores for each grade level tested in Oklahoma. 
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TABLE I 



Percentile Ranks, Stanines, Normal Curve Equivalents, 
and Scaled Scores Corresponding to the Holistic Raw Scores for the 
Stanford Writing Assessment Program, Level: Advanced I 
Form K, for Spring of 1993, Grade 7* 



HOLISTIC RAW 
SCORES 


PERCENTILE 
RANKS 


STA NINES 


NORMAL CURVE 
EQUIVALENTS 


SCALED SCORES 


12 


99 


9 


99.0 


676 


11 


98 


9 


93.3 


666 


10 


95 


8 


84.6 


656 


9 


87 


7 


73.7 


646 


8 


74 


6 


63.5 


636 


7 


57 


5 


53.7 


626 


6 


40 


5 


44.7 


616 


5 


25 


4 


35.8 


606 


4 


13 


3 


26.3 


596 


3 


5 


2 


15.4 


586 


2 


2 


1 


6.7 


576 



TABLE 2 

Percentile Ranks, Stanines, Normal Curve Equivalents, 
and Scaled Scores Corresponding to the Holistic Raw Scores for the 
Stanford Writing Assessment Program, Level: Task 2 
Form K, for Spring of 1993, Grade 10* 



HOLISTIC RAW 
SCORES 


PERCENTILE 
RANKS 


STANINES 


NORMAL CURVE 
EQUIVALENTS 


SCALED SCORES 


12 


98 


9 


93.3 


682 


11 


93 


8 


81.1 


671 


10 


83 


7 


70.1 


660 


9 


69 


6 


60.4 


650 


8 


53 


5 


51.6 


640 


7 


36 


4 


42.5 


630 


6 


23 


4 


34.4 


621 


5 


14 


3 


27.2 


612 


4 


6 


2 


17.3 


603 


3 


2 


1 


6.7 


593 


2 


1 


1 


1.0 


582 



* Administered to Oklahoma's seventh and tenth-grade students in February 1993. 
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Tables 1 and 2 illustrate the effect of spreading a narrow range of scores (such 
as the holistic Raw Score Scale of 2-12) over a larger range of scores (such as 
the Percentile Rank and Normal Curve Equivalent score scales of 1-99). In 
applying such conversions to real scores, (using the Percentile Rank scale as 
an example), it becomes obvious that the small difference produced by 
earning one additional Raw Score point can make a large difference in the 
Percentile Rank increase—especially in the "middle" of the raw score range. 
For example, look at the Percentile Rank associated with a Raw Score of 7 for 
Grade 7, and compare it with the Percentile Rank associated with a Raw Score 
of 8. At this point on the scale, a difference of one Raw Score point means a 
difference of 14 Percentile Rank points. Differences in individuals' or groups' 
norm-referenced scores that are in the middle of the scale often appear to be 
greater than the one or two raw score points would indicate. To avoid the 
harmful effects of such distorted implications, we strongly recommend that 
viewers of test results refrain from comparing scores of individucd students, 
school sites, and/or school districts. 

B. The Analytic Scoring Guidelines: Relationship of Analytic to Holistic Scores 

As was described in Figure 1 (page 20), the Analytic Scoring Guidelines 
included six categories of writing skills to be considered: Ideas and 
Development; Organization, Unity and Coherence; Word Choice; Sentences 
and Paragraphs; Grammar and Usage, and Mechanics. Within each of these 
categories of skills were evaluation criteria representing four levels of student 
performance, with 4 being the highest level. 

Analytic scores are reported (1) at the individual student level by his or her 
performance of 4, 3, 2, or 1 within each of the six categories of skills; and (2) at 
the group level by number and percent of students performing in the 4, 3, 2, 
and 1 performance levels within each skill category. 

The purpose of the analytic scoring technique is to provide data which will 
assist the classroom teacher in determining students' strengths and 
deficiencies in each of the categories of writing skills being measured. These 
data, used in combination with other evidence gathered in regard to students' 
specific needs, will provide a focus for planning writing skill improvement 
programs for individuals and groups of students. 

Guidelines for appropriate interpretation and use of analytic scores are listed 
as follows: 

• Scores for six skill categories are provided (i.e.. Ideas and Development; 
Organization, Unity, and Coherence; Word Choice; Sentences and 
Paragraphs; Grammar and Usage; and Mechanics). During the analytic 
scoring of Oklahoma's papers, each category of skills was considered as 
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independent of the others. Therefore, in their use, scores for the six 
categories are neither to be added to obtain a total score nor averaged to 
determine a mecui analytic score. 

The user of analytic scores is advised to refrain from comparing either the 
individual student's or group's actual performance levels from one skill 
category to another. For example, performance on Sentences and 
Paragraphs should be judged separately to determine (1) the student's 
actual performance level; (2) the deficiencies in skills indicated by that 
performance level (i.e., the existing need); and (3) the instructional 
techniques, activities, and materials necessary to accomplish skill 
improvement. 

Failure to examine performance in each category separately could result in 
ignoring a child's need for additional instruction in a given skill area. 
This error-applied at the group instructional planning level-can result in 
overteaching in one skill category at the expense of instruction in another. 

Holistic and analytic scoring are designed to describe the quality of student 
writing achievement in very different, but complementary ways. The 
holistic readers make a single judgement of the overall merit of the essay. 
This overall merit judgement is based mainly on its organization and its 
content (development of ideas and specific word choice). Sf)elling, writing 
mechanics, and grammar can influence the holistic score if they make the 
student's writing difficult to imderstand. 

The analytic reader looks for evidence of skill and writing proficiency in 
six specific features of the essay. Teachers and parents can use the analytic 
scores for guidance about which aspects of a student's writing are strong 
and which aspects can be improved through instruction. However, 
because the "whole [holistic] is greater than the sum of its parts [analytics]", 
one will not find a perfect relationship between the two scores. 

Nevertheless, there is a positive correlational relationship between the six 
analytic features and the holistic score of students' essays. A study of this 
relationship by The Psychological Corporation obtained the correlations 
presented in Table 3 on page 35. 
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TABLE 3 



OKLAHOMA SCHOOL TESTING PROGRAM 
DIRECT WRITING ASSESSMENT 

Correlation of Holistic and Analytic Scores for the 
Stanford Writing Assessment Program. Second Edition 



ANALYTIC AREA 


GRADE? 


GRADE 10 


L Ideas and Development 


.77 


.77 


IL Organization, Unity and Coherence 


.71 


.71 


III. Word Choice 


.57 


.53 


IV. Sentences and Paragraphs 


.49 


.44 


V. Grammar and Usage 


.41 


.40 


VI. Mechanics 


.41 


.36 



These correlations show that the first three analytic features (Areas I, n, and EQ) are 
more strongly related to students' holistic scores than the final three analytic 
features (Areas IV, V, and VI). This pattern exists at both the seventh and tenth 
grade. The magnitude of the correlations is sHghtly higher ai grade 7. 

A stronger relationship of analytic areas I, EE, and rH to the holistic scores was 
expected. These three analytics deal with the content of the essay and its 
organization and development. The high positive correlations validate the 
contribution of these features to the holistic score. It is also clear that none of the 
first three analytic features totally explains the global holistic rating. The final 
three analytic features (IV, V and VI) are more weakly related to the holistic score 
because these features are of less importance than the first three in explaining the 
overall merit of the essay. 

Another way of viewing the relationship between holistic and analytic scores is 
shown in Tables 4 and 5 (page 37). At each grade level, we have plotted the average 
holistic score earned by students who received an analytic score of 4, 3, 2, or 1. there 
is a separate graph for each analytic area. (See Figures 5 and 8 on pages 38 and 39). 
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analytic scores of 4 and 3 compared with those earning analytic scores of 2 and 1; and 
(2) a wider range of average holistic scores within analytic areas I, II, and HI, 
compared with analytic areas IV, V and VI. Both results were expected. 

In a small number of cases, score inconsistencies did occur. For example, it was 
possible for a student to receive a relatively high holistic score and a low analytic 
score in one or more areas. Conversely, it was possible for a student to receive a 
middle or low holistic score and a high analytic score in one or more areas. 

Each student^s essay is imique and must be judged on its individual merits; one 
must be very cautious about generalizations* Nevertheless, when inconsistencies 
occurred, most often one of the following conditions applied: 

High holistic score/low analytic score. These papers had high overall merit 
(usually the paper was written in a manner that addressed the topic and 
consistentiy remained "on-topic and within the mode of writing" 
throughout, complete with supporting details; exhibited good organization 
and development; etc.), but the paper may have been lacking in one or more 
of the last three analytic areas (sentence variety, structure and length; 
grammar; capitalization, punctuation, and spelling). 

Middle or low holistic score/high analytic score, A paper with these 
characteristics could be a simple reversal of the paper just described-poor 
content development and organization, but excellent use of writing 
mechanics, grammar, and sentence structure. Another possibility is that the 
paper earned a low holistic score because most of the essay was not in the 
correct mode ("off task"). At the same time, several analytic scores could be 
quite high because analytic scoring considers everything that is written, not 
just the parts written in the correct mode. In fact, analytic scores are given to 
papers written to a different topic ("off-topic") as well as "off-task" papers. 
Overall, the holistic and analytic scores of the Stanford Writing Assessment 
show the expected relationship to each other, and give the user important 
information about different qualities of student writing. 
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TABLE 4 

Mean Holistic Raw Score for Each Analytic Score Point 
in Each Analytic Area: Grade 7 



ANALYTIC AREA 


ANALYTIC SCORES 


4 


3 


2 


1 


I. Ideas and Development 


9.92 


7.43 


5.35 


3.17 


II. Organization, Unity and Coherence 


10.05 


7.58 


5.85 


3.89 


III. Word Choice 


9.19 


7.38 


5 74 


3.56 


IV. Sentences and Paragraphs 


8.85 


7.35 


6.04 


5.10 


V. Grammar and Usage 


8.36 


7.06 


6.10 


5.11 


VI. Mechanics 


8.42 


7.19 


6.19 


5.27 



TABLES 

Mean Holistic Raw Score for Each Analytic Score Point 
in Each Analytic Area: Grade 10 



ANALYTIC AREA 


ANALYTIC SCORES 


4 


3 


2 


1 


I. Ideas and Development 


10.22 


8.11 


6.19 


3.41 


II. Organization, Unity and Coherence 


10.10 


8.16 


6.46 


4.08 


III. Word Choice 


9.53 


8.00 


6.38 


3.74 


IV. Sentences and Paragraphs 


9.08 


7.99 


6.90 


5.63 


V. Grammar and Usage 


8.96 


8.11 


7.25 


5.99 


VI. Mechanics 


8.90 


8.01 


6.93 


5.84 
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V. Summary of Results and Interpretation of the Data 

A. Document Returns 

Answer documents from all school districts which serve seventh and 
tenth grade students were returned tor scoring. State norms were 
developed based on this return. 

B. Number Enrolled Versus Number Tested 

Not all students in Grades 7 and 10 were tested due to the following factors: 

• Students who have Individualized Education Plans (lEPs) pursuant to 
Public Law 94-142 were eligible for exemption from OSTP participation 
(see page 3 of this report for a discussion of this issue). If all lEP 
students are exempted from participation in the OSTP, this number can 
constitute up to 12% of the total enrollment. 

• Local school district option was offered in regard to testing students who 
are considered as Limited English Proficient (LEP). If all LEP students are 
exempted from participation in the OSTP, this number can constitute up 
to 2.8% of the total enrollment. 

• Makeup tests were not given to students who v/ere absent during the test 
administration. The average daily absence in Oklahoma schools is 5.5%. 

• Student enrollment is reported on September 15th of each school year. 
These figures can decrease measurably for a number of reasons, especially 
at the high school grades. It is apparent that-beginning at 10th grade- 
dropouts may account for inany of the school leavers. 

Examples of this enrollment phenomenon are provided as follows: The 
Fall 1991 enrollment for Grade 10 was 40,096; the Fall 1992 enrollment for 
this same statewide class of students as 11th graders was 37,011-for a total 
loss of 3,085 students, or 7.7% of the class. The actual dropout rate 
reported for the 10th grade during the 1991-92 school year (i,e., the most 
recent data available) is 5.2%, or 2,076 students. 

An illustration with less pronounced results is found in the comparison 
of Fall 1991 enrollment at Grade 7 (i.e., 44,792) with the Fall 1992 
enrollment figures for this same class of students in 8th Grade (i.e., 44,163). 
The decrease in enrollment of this class is 629 students, or 1.4%. No 
dropout data are available specifically for 7th Grade. 

We will not know how many school leavers are accounted for in the 
difference observed in Fall 1992 10th Grade enrollment and number of 
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students tested in the OSTP Writing Assessment until the Fall 1992 
enrollment and dropout figures are in. However, a certain p>ercent of this 
difference can be attributable to students who are no longer enrolled in 
Oklahoma schools when the tests are administered. 

The differences existing in enrollment figures as compared with number 
of students for whom test results are given in this report are indicated as 
follows: 



Grade Level 
7 

10 

*Source: 
**Scurce: 



Student Enrollment 
FalL 1992* 
4637 
42,189 



Number of 
Students Tested** 
39,661 
33,517 



% of Student 
Enrollment Tested 

85.2% 

79.4% 



State Department of Education Accreditation Reports-Fall 1992 
Maximum number of students reported on the Master List of Test 
Results Summary. (See Appendices B-3 and B-4) 

C Summary of State Level Results 

Copies of state level reports will be found in Appendix B (Master List of Test 
Results Summary: 1987-1993) and Appendices C and D (List of Averages- 
District Level Results). Each of these reports is printed in its entirety. The 
school districts with deleted scores in the List of Averages is due to the testing 
of less than six students. This was done to protect the privacy of those 
students. 



On the whole, the results indicate that Oklahoma's 7th and 10th Grade 
students' writing skill performance levels are estimated to be above the 
national norm. Due to the irrelevance of comparing group level holistic and 
analytic scores, the summary and interpretation of these data are discussed 
categorically as follows: 

1. Normative Data: Holistic Score Summary 

The holistic score average for Oklahoma's 7th grade students who were 
administered the Stanford Writing Assessment was equivalent to a 57th 
national percentile rank. This means that the average 7th grade student in 
Oklahoma performed at a level which was equivalent to and /or better 
than 57% of the students in the national norm group. 

The average holistic score for Oklahoma's 10th grade students was 
equivalent to the 51st national percentile rank. The average 10th grade 
student performed at a level that was equal to or better than 51% of the 
students in the national norm group. Table 6 provides a comparison of 
national norms with Oklahoma students* performance at Grades 7 and 10. 
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TABLE 6 

Comparison of Oklahoma Students' Performance 
with National Norms- 1993 



Type 

of 
Score 


GRADE 7 


GRADE 10 


National 
Norm 


Oklahoma*s 
Performance 


National 
Norm 


Oklahoma*s 
Performance 


Scaled Score 


622.00 


625.5 


639.00 


638.8 


Normal Curve 
Equivalent 


50.00 


63.9 


50.00 


50.7 


Percentile Rank 


50.00 


57 


50,00 


51 


Stanine 


5.00 


5 


5.00 


5 



Each year there are a number of papers that are considered non-scorable due to four 
possible conditions. These conditions, along with codes assigned to each, are 
described as follows: (NS = non-scorable) 

NS-1 = Not scorable because the student refused to write or the student's essay had 
insufficient information, was blank, inappropriate, or a copy of the prompt. 

NS - 2 = Non scorable because the student's essay was ofT- topic (i.e., did not address the topic of 
the prompt) or ofT-task (i.e., was not written in the appropriate writing mode). 

NS - 3 = Not scorable because the student's essay was written in a foreign language. 

NS - 4 = Not scorable because the student's essay was illegible or incomprehensible. 

The 1993 results indicate that in Grade 7, there was a total of 389 non-scorable 
papers; this sum for Grade 10 wao 279. The number occurring in each of the non- 
scorable categories and the percent of total students tested represented by the sum of 
non-scorable papers received are provided as follows: 



Grade Level 


Non-Scorable 

Category 
Designation 


Number of Papers 
Occurring in this 
Category 


% of Papers 
Represented 
by Total 


Grade 7 


NS-1 
NS-2 
NS-3 
NS-4 


76 
307 
1 
5 


.010% 


Grade 10 


NS-1 
NS-2 
NS-3 
NS-4 


58 
219 
1 
1 


.008% 





Table 7 illustrates the distribution of Oklahoma students' scores by providing the 
number and percent of students whose scores occurred in each quarter of the 
percentile rank scale, A comparison with national norms is offered to illustrate the 
difference in "average" performance and Oklahoma students' performance. 
(Appendix A--Percent of Scores Under the Normal Curve-illustrates the distribution 
of "average" group performance,) 

TABLE 7 

Comparison of Oklahoma Students* Performance in 



Each Quartile With That Expected Based on the Normal Curve 



NATIONAL NORMS 


OKLAHOMA STUDENTS' PERFORMANCE 


Percentile 
Rank 




Grade 7-1993 


Grade 10-1993 


% of Scores 


# of Scores 


%of Scores 


# of Scores 


% of Scores 


99 vs^^^^^^ 












76 — ^ 


^25% 


8,609 


22% 


6,882 


21% 














5. 


25% 


13,731 


35% 


12,889 


39% 


50^ 












26 




9,024 


23% 


5,532 


17% 


25 












i ^ 


^ 25% 


7,908 


20% 


7,935 


24% 
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An analysis of the holistic scores indicates that the overall writing 
performance of Oklahoma's students appears to be at levels above the 
average. Our school administrators and teachers are to be commended on 
the provision of instructional programs which produce such results. 

2. Analytic Score Summary 

Figures 7 and 8 (pages 45 through 48) illustrate the performance levels of 
Oklahoma students within each of the categories of writing skills 
measured. Definitions of performance level criteria are provided for each 
skill category with corresponding percentages of students who scored in 
each of the respective performance levels. 

It appears that the majority of students were able to demonstrate 
knowledge and use of basic writing skills as measured by the Stanford 
Writing Assessment . In all, 55%-69% of seventh grade students' 
compositions and 62%-77% of tenth grade students' compositions 
demonstrated capabilities within the two highest levels of performance 
among the various writing skill categories. 
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Figure 7 



State Level Results on Analytic Scoring Criteria 
Stanford Writing Assessment, Grade 7-1993 
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Area I: Ideas and Development 

Extensive development of several ideas (or one, importsint main idea) with 
extension and elaboration on all or most of the points. Look for uniqueness, 
interest to audience, and strong supporting details. Can be exceptional 
writing or extremely thorough. 

Good development of idea with many details extended and elaborated. Ideas 
are fairly well supported. May be an extensive list with little or no elaboration. 

Adequately supported ideas with some details extended or elaborated. 

Weak ideas minimally supported with little or no extension of details. 
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Area 11: Organization, Unity and Coherence 

1 Completely organized with smooth flow from one idea to the next through the 
use of transitions and sequencing. Unity is a strong evident with no wandering 
from the primary theme or plan. 

2 Fairly well organized with good unhy of plan. Some transitions may be used. 
Little or no digression from main ideas or writing mode. 

3 Small amount of organization. Weak plan that may not be well unified. Ideas 
may be only minimally connected. May frequently wander from expected 
writing moda. 

4 Lack of plan or coherence. 
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Area III: Word Choice 



Precise, appropriate, accurate, and specific word choices that convey the 
correct meaning and appeal to the audience. May be vivid and imaginative. 

Word choices that £jre appropriate and specific. May lack * 'sparkle", but 
meaning is clear. 

Fair use of words. May be specific but have little variety. May be simplistic 
and occasionally vague, but generally effective. 

Very simplistic. Meardng may be unclear or inappropriate. 
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Figure 7 
(continued) 

State Level Results on Analytic Scoring Criteria 
Stanford Writing Assessment, Grade 7--1993 

Area IV-Sentences and Paragraphs 

1 ExceUent control and formation of sentences. Variety of sentence structure, 
type, and length contribute to fluency and interest. Paragraphs used where 
appropriate. Few, if any, errors. 

2 Adequate control of sentence formation. Some mix of sentence types, lengths, 
and structures. May contain a small number of errors that do not interfere 
with fluency. May have some attempt at paragraphing. 

3 Most senteuces constructed correctly. May have little variety in type, length, 
or structure. May be somewhat monotonous or choppy. May contain several 
errors or lack control. May have no attempt at paragraphing. 

4 Poor sentence structure with many errors that inhibit fluency or clarity. 
Lacks control. 
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Area V-Grammer and Usage 



50 
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1 Error free or very few errors in approximate proportion to length of paper. 

2 Good grammer a/id word usage. Errors that do not detract from overall 
quality of the paper. 

3 Fa$r grammer and usage. Errors may interfere with meaning. May be 
simplistk. 

4 Poor grammer and word usage with frequent or serious errors. 
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Area VI--Mechanics 

1 Error free or very few errors in approximate proportion to the length of the 
paper. 

2 Most punctuation, capitalization, and spelling correct. Errors do not inter- 
fere with communication. 

3 Some errorsinpunctuation,capltaUzation,and/orspelling that may interfere 
somewhat with communication. 

4 Frequent and/or serious errors in mechanics that interfere with communi- 
cation assigned within each of the analytic score categories. 
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Figure 8 



State Level Results on Analytic Scoring Criteria 
Stanford Writing Assessment, Grade 10--1993 



100 




Area I: Ideas and Development 

1 Extensive development of several ideas (or one, important main idea) with 
extension and elaboration on all or most of the points. Look for uniqueness, 
interest to audience, and strong supporting details. Can be exceptional 
writing or extremely tborough. 

2 Good development of idea with many details extended and elaborated. Ideas 
are fairly well supported. May be an extensive list with little or no elaboration. 

3 Adequately supported ideas with some details extended or elaborated. 

4 Weak ideas minimally supported with little or no extension of details. 
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Area H: Organization, Unity and Coherence 

1 Completely organized with smooth flow f rom one idea to the next through tbe 
use of transitions and sequencing. Unity is a strong evident with no wandering 
from the primary theme or plan. 

2 Fairly well organized with good unity of plan. Some transitions may be used. 
Little or no digression from main ideas or writing mode. 

3 Small amount of organization. Weak plan that may not be well unified. Ideas 
may be only minimally connected. May frequently wander from expected 
writing mode. 

4 Lack of plan or coherence. 
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Area III: Word Choice 



1 Precise, appropriate, accurate, and specific word choices that convey the 
correct meaning and appeal to the audience. May be vivid and imaginative. 

2 Word choices that are appropriate and speciflc. May lack "sparkle", but 
meaning is clear. 

3 Fair use of words. May be specific but have little variety. May be simplistic 
and occasionally vague, but generally effective. 

4 Very simplistic. Meaning may be unclear or inappropriate. 
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Figure 8 
(continued) 

State Level Results on Analytic Scoring Criteria 
Stanford Writing Assessment, Grade 10-1993 

Area IV-Sentences and Paragraphs 

1 00 -J 1 Excellent control and formation of sentences. Variety of sentence structure, 

type, and length contribute to fluency and interest. Paragraphs used where 
appropriate. Few, if any, errors. 

2 Adequate control of sentence formation. Some mix of sentence types, lengths, 
and structures. May contain a small number of errors that do not interfere 

50 -I with fluency. May have some attempt at paragraphing. 

3 Most sentences constructed correctly. May have little variety in type, length, 
orstructure. May be somewhat monotonous or choppy. May contain several 
errors or lack control. May have no attempt at paragraphing. 

4 Poor sentence structure with many errors that inhibit fluency or clarity. 
Lacks control. 
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Area V-Grammer and Usage 



1 Error free or very few errors In approximate proportion to length of paper. 

2 Good grammer and word usage. Errors that do not detract from overall 
49 quality of the paper. 
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3 Fair grammer and usage. Errors may interfere with meaning. May be 
simpllstk. 

4 Poor grammer and word usage with frequent or serious errors. 



Area VI--Mechanics 

1 Error free or very few errors In approximate proportion to the length of the 
paper. 
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2 Most punctuation, capitalization, and spelling correct. Errors do not Inter- 
fere with communication. 

50-1 

3 Some errors in punctuation , capitalization, and/or spelling that may interfere 
somewhat with communication. 
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4 Frequent and/or serious errors in mechanics that interfere with communi- 
cation assigned within each of the analytic score categories. 
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The provision of analytic score data at the individual student level will 
indicate the writing skill area needing the greatest instructional focus and 
practice. Working together with the knowledge base provided by such data, 
parents and teachers can create opportunities for students to become more 
proficient writers. 

At the classroom and building levels, such data can indicate areas of strength 
and deficiency in specific writing skill instruction based on the number and 
percent of students scoring at each of the performance levels within each of 
the skill categories. Such knowledge-put into practice-should serve to 
identify program needs and aid to provide a focus for fulfilling these needs. 

D. Observations 

It appears that, based on the results of the Stanford Writing Assessment, 
Oklahoma's 7th and 10th Grade students were able to organize and construct 
essays in the expository wiiting mode in a manner that compares favorably 
with the performance of the national norm group. 

For most students, it takes many years to develop high quality writing skills. 
This requires quality instruction and ample practice in writing. It appears that 
Oklahoma's students are receiving a good measure of both. 

The analytic score scale was neither designed nor normed to establish the 
expectation that 25% of all students would score in each of the levels (1-4) 
within each analytic score area. However, the four performance levels do 
represent varying degrees of writing expertise-from 1 (low) to 4 (high). It is 
therefore desirable for students to earn one of the two highest analytic scores 
(i.e., either a 4 or 3) in each of the six analytic score areas. 

The following table displays Oklahoma students' analytic scores in terms of 
the combined percent of students earning the analytic scores of 4 and 3 in each 
of the six analytic score areas: 



Combined Percent of Students Eaming 
Analytic Scores of 4 and 3 



Analytic Score Area 



I - Ideas & Development 

n - Organization, Unity, & Coherence 

m - Word Choice 

IV - Sentences & Paragraphs 

V - Grammar & Usage 

VI - Writing Mechanics 



65% 
55% 
65% 
61% 
69% 
65% 



70% 
65% 
77% 
70% 
62% 
73% 
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Comparisons of Stanford Writing Assessment scores earned by the 1992 and 1993 
Oklahoma student groups are only relevant in regard to how each group scored 
relative to national norm group's average (i.e., 50th National Percentile Rank) 
for each prompt utilized. In addition, the descriptive and expository prompts 
used for the 1992 and 1993 assessments, respectively, were not equated during the 
national norming. Therefore, performance of group scores from one prompt to 
another is not directly comparable. 

A study of Oklahoma's 1993 writing assessment data indicates that the 10th grade 
students did not perform as well as (1) the 1993 seventh-grade group; or (2) their 
predecessors (who took the Stanford in 1992) when comparing their holistic score 
means with those of their respective national norm groups. However, the 
analytic score performance of the 10th grade class shows a greater compartive 
proficiency in Word Choice; Sentences and Paragraphs; and Writing Mechanics. 
Performance of the 1993 tenth-grade students was equivalent to that of their 1992 
peers in the Organization, Unity, and Coherence writing skill area. 

The 1993 writing assessment holistic score data indicates that the 7th grade 
students did not perform as well as the 1992 seventh-grade group on the 
Stanford . However, the analytic score performance of these groups indicates that 
the 1993 7th grade class has a greater comparative proficiency in Organization, 
Unity, and Coherence; Word Choice, and Sentence and Paragraphs. 

These results support the recency of Stanford norms as a possible reason for the 
mean holistic score performance level of this year's tenth grade's being so much 
closer to the national average than were their predecessors' scores. The newer 
norms on this test indicate a higher performance average across the nation in 
terms of overall ivriting--i.e., the ability to organize thoughts and apply specific 
writing skills to produce essays that, considered as a whole, represent high 
quality. It appears that Oklahoma's 1993 tenth grade students are being 
compared with a national average that is more difficult to exceed. 

As illustrated above, the 1993 tenth grade scores exhibit good performance in 
each of the six analytic score areas, meaning that Oklahoma students are applying 
each of the specific writing skills well. The level of writing skills demonstrated 
by this analytic score performance will enable our students to seize the 
opportunity for improvement in holistic scores. 

VI. Directions for the Future 

Oklahoma law requires that the writing assessment data be used to improve 
instructional programs in schools throughout the state. The State 
Department of Education will continue to provide a variety of services to aid 
with this process. 



A. Posttest Inservice Training Program 

A posttest inservice training program, will be offered throughout the 1993- 
94 school year for all teachers, building and district test coordinators, and 
school administrators serving middle, junior high, and secondary level 
students. Sessions conducted on-site in each school district will include 
the following topics: 

• Interpretation of the Test Data Reports 

• Analysis of Test Results to Determine Needed Changes (methodology) 

• Specific Teaching Techniques and Activities for Writing and Skill 
Reinforcement 

• Assistance with determining needed changes and innovations in the 
School Improvement Plan 

To prepare for this inservice program. The Psychological Corporation has 
developed materials which are customized to the specific needs of the 
Writing Assessment Component of the OSTP and the follow-up work which 
will take place at the local school level. Such pubUcations include the 
Manual for Interpreting and additional inservice materials which are 
designed to aid schools in using test data at the local level. These materials 
will be provided for each inservice participant. 

B. Follow-up Services Provided for Schools 

At the request of local school adnunistrators, the Department will provide 
assistance to schools based on the stated needs of each faculty. Since the 
implementation of the OSTP (Fall semester, 1985) the number of such 
requests has increased. It is anticipated tl\at a similar interest in use of the 
writing assessment data will be exhibited throughout the 1993-94 school year. 



APPENDIX A 
Percent of Scores Under the Normal Curve 



66 



APPENDIX B 



MASTER LIST OF TEST RESULTS SUMMARY: 1987-1993 



Appendix B-1 Performance of Oklahoma's Students on the MAT-6 
Writing Test: Grade 7 for 1988-91 

Appendix B-2 Performance of Oklahoma's Students on the MATS 
Writing Test: Grade 10 for 1987-1991 

Appendix B-3 Performance of Oklahoma's Students on the Stanford 
Writing Assessment: Grade 7 for 1992-93 

Appendix B-4 Performance of Oklahoma's Students on the Stanford 
Writing Assessment: Grade 10 for 1992-93 
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Performance of Oklahoma's Students on the 
MAT'6 V/riHng Test: Grade 7 
198&-1991 
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Analvtin Srnring Guidplinf*fi 

Area I-Sentence F ormatinTi 

4 Controlled and varied sentence gtructure. Few, if any, sentence formation errors. 

3 Basically adequate sentence structure with little variety in length and form. Few sentence formation 
errors, 

2 Some Errors in sentence foimation. 

1 Frequent and/or serious errors in sentence formation. 

Area n--Word TTsagP 

4 Vocabulary carefully and imaginatively used. Few, if any, errors in grammar. 

3 Vocahulaiy acceptable and appropriate, but somewhat simplistic A few errors in grammar. 

2 i^jpropriate, but limited vocabulary. Some errors in grammar. 

1 Vocabulary limited and at times incorrect or unsuitable. 

Area m-XUmi^t n^volnTvm^p[^ 

4 Well-planned content. Clearly elaborated^ well-organized, controlled ideas. 

3 Several dear and specific details, with a few problems in either oigani2ation or development 

2 Vairue and sketchy details, ofent po<M^y organized. Problems with repetition of ideas and lack of unity. 

1 Weak ideas and rand<Hn thoui^its, pooriy planned. Few details included in a very brief response. 

Area IV-Writing Mpnhflnips 

4 No errors in mechanics, or few errors with some attempt at advanced techniques 

3 Few errors in mechanics; little risk-taking. 

2 Many errors in mechanics. 

1 Frequent and/or serious errors in mechanics, interfering with conununication. 



Appendix B-2 



Performance of Oklahoma's Students on the 
MAT-6 Writing Test: Grade 10 
1987-1991 

Holltrtic Score Results Analytic Score Results 

Performance Levels 



Year 


h'ercentiie KanK 




pcore i\rea 


Q 
fit 








fi 
2 


Jit 


a 


M 


1987 


74 




I 


4 


32% 


3 


48% 


2 


13% 


1 


7% 








n 


4 


43% 


3 


41% 


2 


11% 


1 


5% 








m 


4 


17% 


3 


60% 


2 


21% 


1 


2% 








TV/ 
IV 


A 




Q 
O 


*mJYO 


9 




1 

L 




1988 


(» 


60.7 


I 


4 


16% 


3 


60% 


2 


18% 


1 


6% 








n 


4 


15% 


3 


61% 


2 


20% 


1 


4% 








m 


4 


11% 


3 


49% 


2 


38% 


1 


2% 








IV 


4 




Q 


fit Cf^ 


9 




1 


Xyvc 


1989 


78 




I 


4 


14% 


3 


52% 


2 


24% 


1 


9% 








n 


4 


13% 


3 


60% 


2 


22% 


1 


4% 








m 


4 


13% 


3 


64% 


2 


21% 


1 


2% 








IV 


4 


11% 


3 


44% 


2 


32% 


1 


14% 


1990 


72 


62,2 


I 


4 


25% 


3 


49% 


2 


18% 


1 


8% 








n 


4 


18% 


3 


60% 


2 


18% 


1 


5% 








m 


4 


20% 


3 


60% 


2 


18% 


1 


2% 








IV 


4 


14% 


3 


48% 


2 


24% 


1 


14% 


1991 


78 


66.1 


I 


4 


17% 


3 


65% 


2 


14% 


1 


3% 








n 


4 


14% 


3 


73% 


2 


12% 


1 


1% 








ra 


4 


10% 


3 


78% 


2 


11% 


1 


1% 








IV 


4 


17% 


3 


64% 


2 


15% 


1 


3% 



Analytic Scoring Guidftlines 

Area 1--Sentenn<> Formation 

4 Controlled and varied sentence structure. Few, if any, sentence formation errors. 

3 Basically adequate sentence structure with little variety in length and form. Few sentence 
formation errors. 

2 Some Errors in sentence formatiou. 

1 Frequent and/or serious errors in sentence formation* 

Arpa Il-^Wnrd Usage 

4 Vocabulary carefully and imaginatively used. Few, if any, errors in grammar. 

3 Vocabulary acceptable and appropriate, but somewhat simplistic A few errors in grammar. 

2 Appropriate, but limited vocabulary. Some errors in grammar. 

1 Vocabulaiy limited and at times incorrect or unsuitable. 

Area Ill-^^ntent Devplnnment 

4 Well-planned content Clearly elaborated, well-organized, controlled ideas. 

3 Several clear and speciHc details, with a few problems in either organization or 
development. 

2 Vague and sketchy details, ofent poorly organizedL Problems with repetition of ideas and 
lack of unity. 

1 Weak ideas and random thou^ts, poorly planned. Few details included in a very brief 
response. 

/Vrea TV--Writing Mechanics 

4 No errors in mechanics, or few errors with some attempt at advanced techniques 

3 Few errors in mechanics; little risk-taking. 

2 Many errors in mechanics. 

1 Frequent and/cr serious errors in mechanics, interfering with communication. 

7i. 
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Performance of Oklahoma's Students on the Stanford Writing Assessment 

Grade 7 1992 and 1993 



/ 



ERIC 



S tanford 



HftlTING ASSES3H£KT PROGRAM SECOND EDITION 



SUADE: 07 
LEVEL: INTERhfHO Z 
NORMS: SPUtm FORM : K 



MASTER LIST OF TEST RESULTS 
FOR 

OKLAHOMA 



TYPE Of WRITING: 



HOLISTIC SCORING 

Th« Holistic Raw Scor* is iha slm of two indepondani rat*rs using a 6-poini scoring 
seals » with 6 baing tha highasi. Tha scora rangay tharaforay is 2 to 12. 

ANALYTIC SCORING 
Tha Analytic scoras ara six saparata scorast ona for aach of six araas: 
Idaas and Davalopmant > OrgoniZBtiony Unity> and Coharanoai Hord Choioai 
Santancas and Paragraphs ( Gr aaum r and Usaga} and Machanics. A ^-point 
scoring scala is usad for aach araay with 4 baing tha highest. 



Afi&REVIATIQNS USED: 

Z - NUI«£R> PERCENT 
NAT*L = NATIONAL 
NCE = NORMAL CURVE EQUIVAUhTT 
PR = PERCENTILE RAMC 



RS 
S 

ss 



> RAH SCORE 

r STANINE 

s SCALED SCORE 



FOOTNOTES: 



Statistics do not includa students with non-scorabla assays. 



NA = Analytic soora is not available becausa tha student's essay 
Mas not soorable. 



NAl = Not available because no derived scores can be reported for 
non-scorable essays. 

MSI = Not scorable because the studant refused to write or the studant*s 
assey had insufficient infomationy was blank y inappropriate » 
or a copy of the prosit. 

HS2 s Not scorable because tha studant*s essay Mas off -topic or 
off-task. 



NS3 = Not scorable because the studant's essay was writtan in a 
foreis^ language. 



N$4 s i4ot scorable because the studant's essay mss illegible or 
inooapralians ible . 



Scores besed on Norsiative Date Copyright ^ 19A9 by Karoourt Urace Jovsnovich* Inc. All ri^^ts reserved. 
COPY 1 PROCESS NO. 19396000-CIK95-00001-X 
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HRXnilC ASS£SSHEKr PROCRAH* SECOND E0ITXC3N 



TEST DATE: 02/92 
NORMS: SPRING 



GRADE; 07 
LEVEL: ADVANCED 
FORM : K 



MASTER LIST OF TEST RESULTS 

suraiARY 

FOR 
OKLAHOMA 



TYPE OF WRITING: DESCRIPTIVE 



TOTAL Nl.«ER TESTED 



38372 » 



NON SCORASLE TOTAL 



217 



SCORE TYPE 



HOLISTIC SCORE SUMMARY 
MEAN STANDARD DEVIATION 



MEDIAN 



RAW SCORE 7.1 

SCALED SCORE 62<f.4 

NATIONAL NCE 57.0 

STATE NCE 50.1 

PR-S OF MEAN NAT'L NCE 63-6 

PR-S OF MEAN STATE NCE 50-? 



1.7 
18.9 
16.5 
20.0 



7.2 
620.^ 
52.9 
^5.0 







F 


R E 


Q U E N C Y 


0 I S 


T R 


I B U T 


ION 
















NATIONAL 


STATE 






RAH SCORE 


N-COUNT 


PERCENT 




SCALED SCORE 


PR-S 




NCE 


PR-S 


NCE 






12 


132 


0 




678 


99-9 




99.0 


99-9 


J9.0 






11 


630 


Z 




667 


96-9 




93.3 


99-9 


99,0 






10 


1963 


s 




656 


95-8 




84.6 


95-8 


84.6 






9 


£020 


13 




645 


89-8 




75.8 


86-7 


72.8 






8 


9255 


24 




634 


77-7 




65.6 


68-6 


59.9 






7 


7427 


19 




623 ^ 


60-6 




55.3 


46-5 


47.9 






6 


76L3 


20 




612 


42-5 




45.8 


26-4 


36.5 






5 


31^ 


8 




601 ^ 


^ 26-4 




36.5 


12-3 


25. 3 








2219 


6 




590 






27.2 


5-2 


15.4 






3 


476 


1 




579 






18. 9 


1-1 


1.0 






2 


202 


1 




567 


3-1 




10.4 


1-1 


1.0 






NSl 


45 


0 




















NS2 


170 


0 




















NS3 


0 


0 




















NS4 


2 


0 




















P E 


R C E N T I 


L E S U H 


H A R Y 






S T A 


NINE 


SUM 


K 


A R Y 




NATIONAL 


STATE 










NATIONAL 


STATE 






N 'A 


N 










N 




N 


A 


76 - 99 


17000 45 


7745 


20 




7 - 


9 


17000 


45 


7745 


20 


51 - 75 


7427 19 


9255 


24 




4 - 


6 


18258 


46 


24365 


64 


26-50 


ICB^I 28 


ISllO 


40 




1 - 


3 


2897 


a 


6045 


16 


0-25 


2897 6 


6045 


16 



















ANA 


LYTIC 


SCORE S 


U M M A R Y 


B Y 


AREA 










AREA I 




AREA 


II 


AREA 


III 


AREA 


IV 


AREA V 




AREA VI 




SCORES 


N 


A 


N 


A 


N 


A 


N 


A 


N 


A 


N 


A 


4 


5460 


14 


3767 


10 


3855 


10 


5677 


15 


9220 


24 


7299 


19 


3 


19865 


52 


15954 


42 


18621 


49 


16917 


44 


19229 


50 


18704 


49 


2 


11218 


Z9 


15422 


40 


14545 


353 


12851 


S4 


8533 


22 


9759 


25 


1 


1762 


5 


5182 


8 


1304 


3 


2880 


8 


1343 


4 


2563 


7 
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S tanford 



HRITING ASSESSMEKT PROGIUHr SECOND EDITION 



TEST OATt: 02/95 
NORMS: SPRING 



GRADE: 07 
LEVEL: IWTERflED 
FORM : K 



MASTER LIST OF TEST RESULTS 
SUrtURY 
FOR 
OKLAHOMA 



TYPE OF NRITINC: 



EXPOSITORY 



TOTAL NLMJER TESTED = S9bbl* 



NON SCORABLE TOTAL 



589 







H 0 L 


I S T I C S 


CORE 




S U M M 


A R Y 












SCORE TYPE 


MEAN 


STANDARD DEVIATION 




MEDIAN 






KArl ^<<UKC 








2.0 








6.9 






SCALED SCORE 


625. 5 






19.8 






620.1 






NATIONAL NCE 


53.9 






19.0 






48.2 






STATE NCE 


50.2 






20.2 






44.6 






PR-S OF MEAN NAT'L NCE 


57-5 




















rK o Ur ncAN dlAIC wm^c 






















F R E 


Q U E N C Y 


0 I S 


T R 


I B U T 


ION 
















NATIONAL 


STATE 






RAW SCORE 


N-COUNT PERCENT 


SCALED SCORE 


PR-S 




NCE 


PR-S 


NCE 








12 


510 1 


676 


99-9 




99.0 


99-9 


99.0 








11 


1056 5 


666 


98-9 




95.3 


98-9 


93.3 








10 


2909 7 


656 


95-a 




84.6 


93-8 


81.1 








9 


^55^ 11 


6<V6 


87-7 




73.7 


84-7 


70,9 








8 


6250 16 


636 


74-6 




65.5 


70-6 


61.0 








7 


7<t«l 19 


626 


57-5 




55.7 


55-5 


51.6 








6 


902^ 25 


616 


40-5 




44.7 


32-4 


40.1 








B 


5^10 9 


606 


2S-4 




35.8 


16-3 


29.1 










2957 7 


596 


13-5 




26.5 


6-2 


20.4 








5 


982 5 


586 


5-2 




15.4 


3-1 


10.4 








2 


579 1 


576 


2-1 




6.7 


1-1 


1.0 








NSl 


76 0 




















NS2 


307 1 




















NS5 


1 0 




















NS4 


S 0 




















P E 


RCENTILE SUM 


MARY 






S T A 


NINE 


SUM 


M 


A R Y 






NATIONAL STATE 








NATIONAL 


STATE 










H y, u y. 








N 




N 






76 


- 99 


8609 22 6609 ZZ 




7 - 


9 


8609 


22 


8609 


22 


51 


- 75 


15751 55 13731 55 




4 - 


6 


26165 


67 


22755 


58 


26 


- 50 


902<» 25 9024 25 




1 - 


3 


4498 


11 


7908 


20 


0 


- 25 


7908 20 7908 20 





















ANA 


LYTIC 


SCORE S 


U H M A R Y 


B Y 


AREA 










AREA I 




AREA 


II 


AREA 


III 


AREA 


IV 


AREA V 




AREA VI 




SCORES 


N 


y. 


N 


z 


N 


y 


N 


y 


N 


y 


N 


y 


4 


4694 


12 


3B19 


10 


5737 


9 


4695 


12 


6509 


16 


5205 


15 


5 


21099 


55 


17800 


45 


22285 


56 


19273 


49 


21042 


55 


20494 


52 


2 


12814 


32 


16161 


41 


12770 


52 


15439 


34 


10525 


27 


11479 


29 


1 


972 


2 


1799 


5 


787 


2 


2172 


5 


1503 


4 


2401 


6 
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APPENDIX B-4 

Performance of Oklahoma's Students on the Stanford Wnting Assessment: 

Grade 10 1992 and 1993 



7o 



g TAJslFORD 



HRITINC /ISSESSHEKT PROGRAM » SECOND EDITION 



NORMS: 



SPRING 



GRADE: 10 

LEVEL: ADVANCED 2 

FORM : K 



MASTER LIST OF TEST RESULTS 
FOR 

OKLAHOMA 



TYPE OF KRITING 



HOLISTIC SCORING 

Th. Holistic Ra*. Soor. is th. of two lnd.p..nri«.t raters using a 6-point scoring 
seal*, with 6 baing -U* hij^st. Ih^ ^oorm r«nQ«, th«r«for«, is 2 to 12. 

ANALYTIC SCORING 
Th« Analytic scor«m ar* mix wparata mcor«*, or« for aach of six ar«»: 
Idaas and Davalopii^tj Oryani^atloo, Unity, and Coharanc.^ Word Choic«J 
Sant«.oas and Paragraph* J Sraia-ar and Usages .nd Machanics. A 4-point 
mooring scala Im usad for aach araa, with 4 baing tha hij^st. 



ABBREVIATIOMS USED: 

N, X = NUf«ER» PERCENT 
NAT'L = NATIONAL 
NCE = NORMAL CURVE EQUIVALENT 
PERCENTILE RAMC 



RS 

S 

ss 



= RAH SCORE 

= STANINE 

= SCALFD SCORE 



PR 



FOOTWTES: 



NA 



Statistics do not includa mtudants with non-scorabla assays. 

Analytic scora i« not avaiUbla bacausa tha studant's assay 
Mas not scorabla. 



NAl = Not availabla bacausa no darivad scoras can ba rapcrtad for 
non-scor«bla assays. 

NSl = Not scorabla bacausa tha studant rj^^^** tha stiffs 

assay had insufficiant infon«ticn, was blank, inappropriata, 
OP a copy of tha prompt. 

KS2 = Not soorabla bacausa tha studant 's assay was off -topic or 
off-task. 

NS5 = Not scorabla b«^usa tha studant 's assay was written in a 
foraign lanjM^ga. 

HS^» = Not scorabla b^sausa tha stodanfs assay was illagibla or 
inooa^ir^hans ibla . 
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S tanford 



NRITINe ASSESShEKT PR0C1UM» SECOND EDITION 



TEST DATE: 

NORMS: SPRING 



GRADE: ID 
LEVEL: TASK 2 
FORM : K 



MASTER LIST OF TEST RESULTS 
SUmARY 
FOR 
OKLAHOMA 



TYPE OF HRITING: 



DESCRIPTIVE 



TOTAL NUtBER TESTED = 52568* 



NON SCORABLE TOTAL = 



225 



SCORE TYPE 



HOLISTIC 
MEAN 



SCORE SUMMARY 
STANDARD DEVIATION 



MEDIAN 



RAN SCORE 7.6 

SCALED SCORE 6^2.8 

NATIONAL NCE 53.5 

STATE NCE 50.0 

PR-S OF MEAN NAT'U NCE 57-B 

PR-S OF MEAN STATE MCE 50-S 



1.9 
19.8 
17.0 
20.3 



7.7 
638.5 

V:.7 



FREQUENCY 



DISTRIBUTION 











NATIONAL 


STATE 




RAH SCORE 


N~COUNT 


PERCENT 


SCALED SCORE 


PR-S 


NCE 


PR-S 


NCE 




12 


52^ 


2 


694 


99-9 


99.0 


99-9 


99.0 




11 


1455 


5 


681 


96-9 


86.9 


96-9 


86.9 




10 


2652 


e 


669 


90-8 


77.0 


90-e 


77.0 




9 


466a 


15 


657 


77-7 


65.6 


78-7 


66.3 




8 


8272 


26 


64« 


61-6 


55.9 


56-5 


54.2 




7 


6119 


19 


636 


44-5 


46.8 


35-4 


41.9 




6 


4408 


14 


626 


30-4 


39.0 


19-3 


31.5 




5 


2035 


£ 


616 


19-3 


31.5 


9-2 


21.8 






1175 


4 


606 


10-2 


23.0 


4-2 


13.1 




3 


415 


1 


596 


5-2 


15.4 


1-1 


1.0 




2 


242 


1 


£86 


2-1 


6.7 


1-1 


1,0 




NSl 


67 


0 














NS2 


154 


0 














NS3 


1 


0 














NS4 


1 


0 














P E 


R C E N T I 


LE SUMMARY 




S T A 


NINE 


3 U M M 


A R Y 




NATIONAL 


STATE 






NATIONAL 


STATE 




N y. 


N y 






N 


'A 


N y 


76 - 99 


9499 30 


9499 30 




7-9 


9499 


30 


9499 30 


51-75 


8272 26 


8272 26 




4-6 


18799 


58 


14391 45 


26 - 50 


10527 S3 


6119 19 




1-3 


3867 


12 


8275 26 


0-25 


3867 12 


8275 26 















ANA 


LYTIC 


SCORE S 


U M M A R Y 


B Y 


AREA 










AREA I 




AREA 


II 


AREA 




AREA 


IV 


AREA V 




AREA VI 




SCORES 


N 


V. 


N 


'A 


N 






X 


N 


V. 


N 


X 


4 


5511 


17 


4518 


14 


4248 


13 


6050 


19 


8520 


26 


8052 


25 


3 


18750 


58 


16349 


51 


19905 


62 


15O40 


47 


15121 


47 


14675 


45 


2 


7325 


23 


10604 


33 


7586 


23 


9308 


29 


7413 


23 


rm 


24 


1 


735 


2 


848 


3 


580 


2 


1921 


6 


1265 


4 


1815 


6 
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HRITINe ASSESSMCKT PROGRAM, SECOND EDITION 



TEST DATE: 02/95 
NORMS: SPRING 



GRACE: 10 
LEVEL: ADVANCED 
FORM : K 



MASTER LIST OF TEST vlESULTS 
SUflARY 
FOR 
OKLAHOMA 



TYPE 



0 F 



N R I T I N 6 



EXPOSITORY 



TOTAL NI>BER TESTED = 53517* 



NON SCORABLE TOTAL 



279 







H 0 


L 


I S T I C S 


CORE 




S U H H 


A R Y 








SCORE TYPE 






MEAN 


STANDARD DEVIATION 


MEDIAN 




RAH SCORE 






7.8 






2.0 






7.9 




SCALED SCORE 




658.8 






19.1 




654.9 




NATlOt^L NCE 




50.7 






17.7 




46.7 




STATE NCE 






^9.9 






20.5 




45.0 




PR-S OF MEAN NAT'L NCE 


51-5 
















PR-S OF MEAN STATE 


NCE 


50-5 


















F 


R E 


Q U E N C Y 


0 I S 


T R 


I B U T 


ION 
















NATIONAL 


STATB 






N-CDUNT 


PERCENT 




SCALED SCORE 


PR-S 




NCE 


PR-S 


NCE 




12 


675 


2 




682 


96-9 




9S.5 


99-9 


99.0 




11 


1592 


5 




671 


93 -a 




81.1 


96-9 


66.9 




10 


^615 


1^ 




660 


83-7 




70,1 


86-7 


72.8 




9 


5692 


17 




650 


69-6 




60.4 


71-6 


61.7 
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APPENDIX C 
LIST OF AVERAGES-DISTRICT LEVEL RESULTS: 

Grade 7 - 1993 
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APPENDIX D 
LIST OF AVERAGES-DISTRICT LEVEL RESULTS: 

Grade 10 - 1993 
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